As an e-commerce platform, we provide a page for users to choose a pickup point when they place an order. One afternoon, my colleague suddenly found that this page would request the API again. The phenomenon was:
- First, a Pre-flight/OPTIONS request (because the API is cross-domain), but the request is failed. From the F12 Network panel, you can only see
net::ERR_FAILED, and you can't see the detailed error information. There is no error in the console as well. - Then a normal Pre-flight/OPTIONS request, the request is successful.
- Then a normal GET request, the request is successful.
Every API has this problem, so it should be a common problem.
I tried it on my computer and result in a common conclusion of senior programmers: there is no problem on my computer, it's your problem, you see I have screenshots as evidence... The screenshot is the same as the above picture except for the red record.
A Problem That Can't Be Seen in F12
Of course, joking aside, this problem can be reproduced on more than one colleague's computer, so I still have to check it carefully. Common request problems can be checked directly with the F12 Network panel, but this error seems to leave no trace.
The data in the F12 Network panel is parsed, some very low-level content (such as the Connect request header) cannot be seen. The page uses HTTPS, so Wireshark seems to be unable to capture valid data.
Does Chrome itself have a place to view low-level data? Yes! Chrome's NetLog can record the browser's network events completely, and then we can use the official NetLog Viewer tool to view it.
NetLog is a Godsend
NetLog is a low-level debugging tool provided by Chrome. Open the chrome://net-export/ page, you can see that the design is very primitive:
There is only one useful button: "Start Logging to Disk", which is used to save network events to a file on the disk. There are also some options below:
- Strip private information: Remove all private information. This is the default option. If you want to send the saved file to the Internet, it is best to use only this option to prevent sensitive information leakage.
- Include cookies and credentials: Record cookies and other credentials.
- Include raw bytes: Record the raw data of requests and responses. This option is very useful for troubleshooting, but it will make the file very large, and it also contains cookies and other credentials.
In order to prevent the file from being too large, there is an input box at the end to limit the file size (in MB). When the file size exceeds this size, the recording will stop automatically. If not filled, the size will not be limited.
After clicking "Start Logging to Disk" and selecting the disk location, Chrome will record network events in the background. During this process, you can use Chrome normally. If you want to stop recording, you can click the Stop Logging button on the page (if you accidentally close this page, just open it again).
After stopping, you can see two buttons: "Show File" (will open the file manager) and "Start Over" (back to the first step).
Getting Started with NetLog Viewer
The generated file can be viewed in NetLog Viewer. Select the generated file on the homepage (or drag the file in), the page will automatically analyze the file content and display some basic information.
There are many options on the left menu, which can let you view various information, the more commonly used are:
- Events: All network events, including requests, responses, redirects, DNS resolution, TCP connections, SSL handshakes, etc.
- DNS: DNS configuration, resolution results for domains in the record.
- HTTP/2: HTTP/2 frames.
- QUIC: QUIC frames.
- Modules: Includes Chrome extensions, as well as some information on Windows (Layered Service Providers and Namespace Providers).
At this point, we can see that the data we want to see is in Events.
Data in Events
At first glance, the Events page may seem a bit aimless, but if you have used Wireshark or Fiddler before, you will find that the usage is basically the same:
Find the Required Events
First, enter the filtering conditions after the blue question mark. For example, if we are only interested in cross-domain requests on the business page, we can directly enter the URL (only enter the domain is also okay, because it is a fuzzy match). You can see that there is the request information we are concerned about:
You can see some specific information on the right by clicking the record directly (or check the checkbox in the first column to view multiple records on the right). For example, I checked the record with ID 943078 (the first URL_REQUEST record), you can see the detailed information of this request (sensitive data is replaced by [MASKED]):
t=33855698 [st= 0] +CORS_REQUEST [dt=527]
--> cors_preflight_policy = "consider_preflight"
--> headers = "sec-ch-ua: \"Google Chrome\";v=\"113\", \"Chromium\";v=\"113\", \"Not-A.Brand\";v=\"24\"\r\nContent-Type: application/json\r\nDNT: 1\r\nsec-ch-ua-mobile: ?0\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36\r\nX-Request-ID: e1da6-b793-c5f1-b3a\r\nsec-ch-ua-platform: \"macOS\"\r\nAccept: */*\r\n\r\n"
--> is_revalidating = false
--> method = "POST"
--> url = "https://[MASKED]/get_location_children/"
t=33855698 [st= 0] CHECK_CORS_PREFLIGHT_REQUIRED
--> preflight_required = true
--> preflight_required_reason = "disallowed_header"
t=33855698 [st= 0] CORS_PREFLIGHT_URL_REQUEST
--> source_dependency = 943079 (URL_REQUEST)
t=33856115 [st=417] CORS_PREFLIGHT_ERROR
--> cors-error = 26
--> error = "ERR_FAILED"
t=33856115 [st=417] CHECK_CORS_PREFLIGHT_REQUIRED
--> preflight_required = true
--> preflight_required_reason = "private_network_access"
t=33856115 [st=417] CORS_PREFLIGHT_URL_REQUEST
--> source_dependency = 943119 (URL_REQUEST)
t=33856162 [st=464] CORS_PREFLIGHT_RESULT
--> access-control-allow-headers = "content-type,x-request-id"
--> access-control-allow-methods = "GET,POST,PUT"
t=33856162 [st=464] +REQUEST_ALIVE [dt=63]
--> priority = "MEDIUM"
--> traffic_annotation = 101845102
--> url = "https://[MASKED]/get_location_children/"
t=33856162 [st=464] NETWORK_DELEGATE_BEFORE_URL_REQUEST [dt=0]
t=33856162 [st=464] +URL_REQUEST_START_JOB [dt=62]
--> logtiator = "https://[MASKED]"
--> load_flags = 66 (BYPASS_CACHE | DO_NOT_SAVE_COOKIES)
--> method = "POST"
--> network_isolation_key = "https://[MASKED] https://[MASKED]"
--> request_type = "other"
--> site_for_cookies = "SiteForCookies: {site=https://[MASKED]; schemefully_same=true}"
--> upload_id = "0"
--> url = "https://[MASKED]/get_location_children/"
t=33856162 [st=464] COMPUTED_PRIVACY_MODE
--> privacy_mode = "enabled"
t=33856162 [st=464] NETWORK_DELEGATE_BEFORE_START_TRANSACTION [dt=1]
t=33856163 [st=465] HTTP_CACHE_GET_BACKEND [dt=0]
t=33856163 [st=465] +HTTP_STREAM_REQUEST [dt=0]
t=33856163 [st=465] HTTP_STREAM_JOB_CONTROLLER_BOUND
--> source_dependency = 943127 (HTTP_STREAM_JOB_CONTROLLER)
t=33856163 [st=465] HTTP_STREAM_REQUEST_BOUND_TO_JOB
--> source_dependency = 943128 (HTTP_STREAM_JOB)
t=33856163 [st=465] -HTTP_STREAM_REQUEST
t=33856163 [st=465] +URL_REQUEST_DELEGATE_CONNECTED [dt=0]
t=33856163 [st=465] LOCAL_NETWORK_ACCESS_CHECK
--> client_address_space = "public"
--> resource_address_space = "local"
--> result = "allowed-by-target-ip-address-space"
t=33856163 [st=465] -URL_REQUEST_DELEGATE_CONNECTED
t=33856163 [st=465] UPLOAD_DATA_STREAM_INIT [dt=0]
--> is_chunked = false
--> net_error = 0 (?)
--> total_size = 36
t=33856163 [st=465] +HTTP_TRANSACTION_SEND_REQUEST [dt=0]
t=33856163 [st=465] HTTP_TRANSACTION_HTTP2_SEND_REQUEST_HEADERS
--> :method: POST
:authority: [MASKED]
:scheme: https
:path: /[MASKED]/get_location_children/
content-length: 36
pragma: no-cache
cache-control: no-cache
sec-ch-ua: "Google Chrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
content-type: application/json
dnt: 1
sec-ch-ua-mobile: ?0
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
x-request-id: e1da6-b793-c5f1-b3a
sec-ch-ua-platform: "macOS"
accept: */*
origin: https://[MASKED]
sec-fetch-site: same-site
sec-fetch-mode: cors
sec-fetch-dest: empty
referer: https://[MASKED]/
accept-encoding: gzip, deflate, br
accept-language: en-GB,en;q=0.9,zh-HK;q=0.8,zh;q=0.7,en-US;q=0.6,zh-CN;q=0.5
sec-gpc: 1
t=33856163 [st=465] UPLOAD_DATA_STREAM_READ [dt=0]
--> current_position = 0
t=33856163 [st=465] HTTP2_STREAM_UPDATE_SEND_WINDOW
--> delta = -36
--> stream_id = 5
--> window_size = 65500
t=33856163 [st=465] -HTTP_TRANSACTION_SEND_REQUEST
t=33856163 [st=465] +HTTP_TRANSACTION_READ_HEADERS [dt=60]
t=33856223 [st=525] HTTP2_STREAM_UPDATE_SEND_WINDOW
--> delta = 2147418111
--> stream_id = 5
--> window_size = 2147483611
t=33856223 [st=525] HTTP_TRANSACTION_READ_RESPONSE_HEADERS
--> HTTP/1.1 200
server: SGW
date: Thu, 01 Jun 2023 06:11:01 GMT
content-type: application/json; charset=utf-8
vary: Accept-Encoding
access-control-allow-origin: https://[MASKED]
lpsreply: True
ret-code: 0
x-request-id: |e1da6-b793-c5f1-b3a|bvvX
content-encoding: gzip
t=33856223 [st=525] -HTTP_TRANSACTION_READ_HEADERS
t=33856223 [st=525] +NETWORK_DELEGATE_HEADERS_RECEIVED [dt=1]
t=33856223 [st=525] HTTP2_STREAM_UPDATE_RECV_WINDOW
--> delta = -326
--> stream_id = 5
--> window_size = 6291130
t=33856224 [st=526] -NETWORK_DELEGATE_HEADERS_RECEIVED
t=33856224 [st=526] URL_REQUEST_FILTERS_SET
--> filters = "GZIP"
t=33856224 [st=526] -URL_REQUEST_START_JOB
t=33856224 [st=526] URL_REQUEST_DELEGATE_RESPONSE_STARTED [dt=0]
t=33856224 [st=526] HTTP_TRANSACTION_READ_BODY [dt=0]
t=33856224 [st=526] URL_REQUEST_JOB_BYTES_READ
--> byte_count = 326
--> bytes =
1F 8B 08 00 00 00 00 00 04 03 8D 93 4F 4B C3 30 ... ....OK.0
18 C6 EF FD 14 21 67 0F 69 D3 BA D6 AF 22 22 A1 .....!g.i...."".
[MASKED] [MASKED]
t=33856224 [st=526] URL_REQUEST_JOB_FILTERED_BYTES_READ
--> byte_count = 1072
--> bytes =
7B 0A 20 22 72 65 74 63 6F 64 65 22 3A 20 30 2C {. "retcode": 0,
0A 20 22 6D 65 73 73 61 67 65 22 3A 20 22 73 75 . "message": "su
[MASKED] [MASKED]
t=33856225 [st=527] HTTP_TRANSACTION_READ_BODY [dt=0]
t=33856225 [st=527] -CORS_REQUEST
t=33856225 [st=527] -REQUEST_ALIVEIt looks complicated, but in fact it is traceable.
Common Concepts of Events
First of all, let's understand a concept. The events here are actually a collection of transactions and events in the monitoring:
Event refers to an "event"
- For example, output an error message, verify a data.
- The event only has a few fields such as type, occurrence time, carried data, and the transaction it belongs to.
Transaction refers to a "process" with start and end time
- For example, from the beginning to the end of a request, the loading process of a page.
- A transaction can contain multiple transactions or events, for example, a page loading transaction may contain API requests (transaction), error (event), CORS verification (event), etc.
- Transaction has a duration field compared to events, and some systems will save "start/stop time" instead of "start/duration time", which is essentially the same.
Through the combination of transactions and events, we can form a message tree, or use a flame graph to display the tree, which is used to locate performance problems (a common technique is that performance problems often occur at the widest nodes at the bottom).
Text Format of Events
Let's take a look at the format of each line:
t=browser time [st=start time] +Type A [dt=duration]
--> Carried data, represented by key = value
--> Only one key = value per line
--> Each "+Type" corresponds to a "-Type" (transaction end)
--> Sometimes there will be data like "--> source_dependency = 943079 (type)"
--> You can click on the corresponding event to view its detailIt looks much clearer now.
Find the Problem
I exported an events on both my computer and my colleague's computer. After comparison, I found that there was a difference - this is the situation on my colleague's computer (the problem can be reproduced):
t=33855698 [st= 0] CHECK_CORS_PREFLIGHT_REQUIRED
--> preflight_required = true
--> preflight_required_reason = "disallowed_header"
t=33855698 [st= 0] CORS_PREFLIGHT_URL_REQUEST
--> source_dependency = 943079 (URL_REQUEST)
t=33856115 [st=417] CORS_PREFLIGHT_ERROR
--> cors-error = 26
--> error = "ERR_FAILED"
t=33856115 [st=417] CHECK_CORS_PREFLIGHT_REQUIRED
--> preflight_required = true
--> preflight_required_reason = "private_network_access"
t=33856115 [st=417] CORS_PREFLIGHT_URL_REQUEST
--> source_dependency = 943119 (URL_REQUEST)
t=33856162 [st=464] CORS_PREFLIGHT_RESULT
--> access-control-allow-headers = "content-type,x-request-id"
--> access-control-allow-methods = "GET,POST,PUT"
; source_dependency = 943079 (URL_REQUEST)
t=33856114 [st=416] +URL_REQUEST_DELEGATE_CONNECTED [dt=0]
t=33856114 [st=416] LOCAL_NETWORK_ACCESS_CHECK
--> client_address_space = "public"
--> resource_address_space = "local"
--> result = "blocked-by-policy-preflight-warn"
t=33856114 [st=416] -URL_REQUEST_DELEGATE_CONNECTED
; source_dependency = 943119 (URL_REQUEST)
t=33856116 [st= 1] +URL_REQUEST_DELEGATE_CONNECTED [dt=0]
t=33856116 [st= 1] LOCAL_NETWORK_ACCESS_CHECK
--> client_address_space = "public"
--> resource_address_space = "local"
--> result = "allowed-by-target-ip-address-space"
t=33856116 [st= 1] -URL_REQUEST_DELEGATE_CONNECTEDWe can see that the process on my colleague's computer is as follows:
CHECK_CORS_PREFLIGHT_REQUIREDCheck whether a pre-flight request is required, because there is a special header, according to the CORS specification, a pre-flight request needs to be sent firstCORS_PREFLIGHT_URL_REQUESTThe content related to this pre-flight requestURL_REQUEST_DELEGATE_CONNECTEDThe process of sending the requestLOCAL_NETWORK_ACCESS_CHECKCheck local network access, the conclusion isblocked-by-policy-preflight-warn
CORS_PREFLIGHT_ERRORThe pre-flight request failed, the error code is 26, and the error message isERR_FAILEDCHECK_CORS_PREFLIGHT_REQUIREDCheck whether a pre-flight request is required again, the conclusion is yes, because ofprivate_network_accessCORS_PREFLIGHT_URL_REQUESTThe content related to this pre-flight requestURL_REQUEST_DELEGATE_CONNECTEDThe process of sending the requestLOCAL_NETWORK_ACCESS_CHECKCheck local network access, the conclusion isallowed-by-target-ip-address-space
CORS_PREFLIGHT_RESULTThe pre-flight request was successful and the allowed headers and methods were returned
And the record on my computer (cannot reproduce the problem) is like this:
t=41646247 [st= 0] CHECK_CORS_PREFLIGHT_REQUIRED
--> preflight_required = true
--> preflight_required_reason = "disallowed_header"
t=41646247 [st= 0] CORS_PREFLIGHT_URL_REQUEST
--> source_dependency = 1007510 (URL_REQUEST)
t=41646467 [st=220] CORS_PREFLIGHT_RESULT
--> access-control-allow-headers = "content-type,x-request-id"
--> access-control-allow-methods = "GET,POST,PUT"
; source_dependency = 1007510 (URL_REQUEST)
t=41646248 [st= 1] +URL_REQUEST_DELEGATE_CONNECTED [dt=0]
t=41646248 [st= 1] LOCAL_NETWORK_ACCESS_CHECK
--> client_address_space = "loopback"
--> resource_address_space = "unknown"
--> result = "allowed-no-less-public"
t=41646248 [st= 1] -URL_REQUEST_DELEGATE_CONNECTEDThe corresponding process is as follows:
CHECK_CORS_PREFLIGHT_REQUIREDCheck whether a pre-flight request is required, because there is a special header, according to the CORS specification, a pre-flight request needs to be sent firstCORS_PREFLIGHT_URL_REQUESTThe content related to this pre-flight requestURL_REQUEST_DELEGATE_CONNECTEDThe process of sending the requestLOCAL_NETWORK_ACCESS_CHECKCheck local network access, the conclusion isallowed-no-less-public
While digging into the problem, I found five questions that are beyond my knowledge:
- What is
LOCAL_NETWORK_ACCESS_CHECK? - What is
private_network_access? Why does it cause the browser to send a pre-flight request? - Why does the browser send a pre-flight request again after the first pre-flight request fails, instead of failing directly?
- What is
allowed-by-target-ip-address-space? - What is
allowed-no-less-public? Why is my parameter and conclusion different from my colleague's in the check?
Well, well, there are a lot of questions, let's start with one by one.
What is Private Network Access
After some searching, I found an article on the Chrome team blog: Private Network Access update: Introducing a deprecation trial, which describes this feature in detail, and gives a link to the related specification (this specification has been renamed to Local Network Access, for convenience, it is referred to as PNA below).
The concepts and examples in this section are taken from the specification, I modified them slightly, and added some personal understanding.
An Example of Using an Intranet Vulnerability
In RFC 1918 in 1996, private addresses (local, LAN) and public addresses (Internet) have been clearly defined. For some reasons, the security protection of the industry is usually more targeted at external systems, and ignores the protection of intranet systems. Therefore, some intranet devices such as routers, printers, etc., not only have a web management interface, but also usually do not have good security protection; some internal management platforms of the company can also execute some operations without login.
In addition, the UA (usually the browser) does not make too much distinction between the two, which will cause a problem - pages from the Internet can initiate requests to intranet domains:
<!-- A page from https://evil.com -->
<iframe href="https://admin:[email protected]/set_dns?server1=123.123.123.123">
</iframe>This request will be sent to the router by UA, but the purpose of it is to change the router's DNS settings, which is a dangerous behaviour and may lead to the MITM attack. Since iframe is used, CORS is not helpful, and the SameSite policy of Cookie can not work either.
Three Types of IP Addresses
IP addresses can be divided into three categories:
- Loopback, such as
127.0.0.1,::1, etc. Interestingly, according to the IPv4 standard, the entire127.0.0.1/8subnet belongs to the loopback which points to the local machine, so generally it can be pinged even without installing a network card. - Local, such as common
192.168.xx.yy,10.xx.yy.zz, etc. If the devices are in different LANs, these IPs will point to different places. - Public, such as IP on the Internet.
The "public" attribute of these three types of IP decreases from top to bottom, for example, public is more "public" than local, and local is more "public" than loopback.
The Idea of PNA
Since for some reasons, the security of intranet devices is generally lower, then according to the "default security" of the top ten principles of security design, as long as the browser finds that the IP of the current page is more "public" than the requested IP, it will try to send a pre-flight request. If the target allows the current page to send a request (can be judged by the Origin header), it can return the Access-Control-Request-Private-Network: true response header to explicitly agree to this request.
Let's take a look at the detection process in the log:
LOCAL_NETWORK_ACCESS_CHECK
--> client_address_space = "public"
--> resource_address_space = "local"
--> result = "blocked-by-policy-preflight-warn"We can see that client_address_space is the IP address type of the current page, resource_address_space is the IP address type of the request, and result is the result of the check. In the first check on my colleague's computer, the requested IP is more private, so it is blocked.
But this did not solve the problem. The domain of the cross-domain request is also an external domain. Why is it judged as local? In addition, the problems of the other two checks have not been answered:
; Why is the result different from the previous check even with the same parameters?
LOCAL_NETWORK_ACCESS_CHECK
--> client_address_space = "public"
--> resource_address_space = "local"
--> result = "allowed-by-target-ip-address-space"
; Why is the IP address type changed on my computer?
LOCAL_NETWORK_ACCESS_CHECK
--> client_address_space = "loopback"
--> resource_address_space = "unknown"
--> result = "allowed-no-less-public"The Impact of Intranet DNS
Both my colleague and I pinged the domain of the cross-domain request and found that it was resolved to 10.131.xx.yy, which is an intranet IP address...
Interesting, I went to the company's platform to find the record of this domain, and found that it has a private DNS record, which is a CNAME to another domain, and the latter uses an A record to point to this IP. In addition, the former also has a public DNS record, which points to our gateway to the Internet.
It is reasonable to use intranet IP in the intranet to speed up access and reduce the pressure on the gateway, but not all domains are configured with private DNS, which causes the problem of PNA. This will not affect external users.
The problem of "judged as local" has been answered, and there are still a few questions left. NetLog and the specification are not enough, we still need to see how Chrome implements it. Fortunately, the field name is special, and it should not be too difficult to search in the source code.
Implementation in Chrome
Trying to find out how Chrome checks the PNA rules, I searched for client_address_space in the Chromium source code and found a third_party/blink/common/security/address_space_feature.cc file with a symbol definition. Clicking on the symbol shows all the references:
There are a lot of things like AddressSpace::kLoopback, AddressSpace::kLocal, AddressSpace::kPublic, AddressSpace::kUnknown, which should be the enumeration values of IP address types. Searching for AddressSpace::kLocal lead to a suspicious function LocalNetworkAccessChecker::Check, which calls LocalNetworkAccessChecker::CheckInternal, and the latter is the detailed check logic. You can see that the return value is just the result field in NetLog, for example:
return Result::kBlockedByLoadOption;
return Result::kAllowedPotentiallyTrustworthySameOrigin;
return Result::kAllowedMissingClientSecurityState;
return Result::kAllowedByPolicyAllow;
return Result::kAllowedByTargetIpAddressSpace;
// ...That's it! I directly looked into Result::kBlockedByPolicyPreflightWarn which corresponds to blocked-by-policy-preflight-warn, and found it at the end of the function. It looks like the conclusion of "not passed" has been drawn, and it needs to be determined how to handle it according to the current policy of the browser. The current policy of the browser is Policy::kPreflightWarn, so this error is returned:
switch (policy) {
case Policy::kAllow:
NOTREACHED(); // Should have been handled by the if statement above.
return Result::kAllowedByPolicyAllow;
case Policy::kWarn:
return Result::kAllowedByPolicyWarn;
case Policy::kBlock:
return Result::kBlockedByPolicyBlock;
case Policy::kPreflightWarn:
return Result::kBlockedByPolicyPreflightWarn;
case Policy::kPreflightBlock:
return Result::kBlockedByPolicyPreflightBlock;
}That is to say, the detection logic should be in the previous code, as long as the detection is passed, a value like Result::kAllowXXX will be returned. Then I will take a look at how allowed-by-target-ip-address-space passed the check on my colleague's computer. This is the only code that returns this result:
// If target_address_space_ is not Unknown, it will check whether it is the same as resource_address_space
// On my colleague's computer, it must not be Unknown (otherwise it won't get here)
// Since resource_address_space is local in the log, it is to check whether it is equal to kLocal
if (target_address_space_ != mojom::IPAddressSpace::kUnknown) {
if (resource_address_space == target_address_space_) {
return Result::kAllowedByTargetIpAddressSpace;
}
// ...
}Another new term popped up: target_address_space_. After tracing the code, I found Request::CreateRequestWithRequestOrString and ResourceRequestHead::SetTargetAddressSpace functions, and the latter needs my attention. Clicking on this symbol, I see that most of the calls pass kUnknown as the parameter (although the 2nd, 3rd, and 7th records do not indicate kUnknown, but it is only because the parameter is on the next line, and it is also kUnknwon after clicking in), which also shows that for most requests such as frame, xhr, manifest, etc., they will not be passed by this logic because kUnknown is set.
After excluding these records, only the first and last records are left, but after continuing to trace, I found that the call chain has returned...
Definition in the Specification
When I encounter a bottleneck while reading the code, I will go back to the specification to find the answer. In the 3.1.2 CORS preflight section of the PNA specification, I saw several descriptions related to target IP address space (this is not all the content, only excerpts of related paragraphs):
- 1. Add a new target IP address space property to the
requeststruct, initiallynull. 2. Amend the Local Network Access check algorithm to handle this new property...
2.1. If
request's target IP address space is notnull, then:- 2.1.1. If
connection's IP address space is not equal to therequest's target IP address space, then return anetwork error. - 2.1.2. Return null.
- 2.1.1. If
- 2.2. Let
clientAddressSpaceberequest's policy container's IP address space. 2.3. If
response's IP address space is less public thanclientAddressSpace, then:- 2.3.1. Let error be a
network error.
- 2.3.1. Let error be a
3. Define a new algorithm called
HTTP-no-service-workerfetch based on the existing steps in HTTP fetch...3.1. At the very start:
- 3.1.1. If
request's target IP address space is notnull, then setmakeCORSPreflighttotrue.
- 3.1.1. If
3.2. Immediately after running CORS-preflight fetch:
3.2.1. If
preflightResponseis anetwork error:- 3.2.1.1. If
preflightResponse's IP address space is null, returnpreflightResponse. - 3.2.1.2. Set
request's target IP address space topreflightResponse's IP address space. - 3.2.1.3. Return the result of running
HTTP-no-service-workerfetch givenfetchParams.
- 3.2.1.1. If
At first glance, everything seems to be normal, but if you look closely, you will find that the process of HTTP-no-service-worker is defined in 3, and the last line 3.2.1.3 is a recursive call! And this is the reason for the two pre-flight requests. Organize the ideas, this is what happened on my colleague's computer:
fetch(request)request's target IP address space isnull, no processing is performed (rule 3.1)[First OPTIONS]
preflightResponse = fetch(innerRequest)(a pre-flight request is sent due to CORS)innerRequest's target IP address space isnull, no processing is performed (rule 3.1)innerRequest's target IP address space isnull, no processing is performed (rule 2.1, I did not write the repetition here, it's just two identical judgments)clientAddressSpaceis assigned to the IP address space ofinnerRequest's policy container (by default, it isclientaccording to the specification), which ispublicin the log (rule 2.2)response's IP address space islocal, less thanclientAddressSpace, so returnnetwork error(rule 2.3)
It is found that
preflightResponseisnetwork error(rule 3.2.1)preflightResponse's IP address space islocal, notnull, so browser setsrequest's target IP address space tolocal(rule 3.2.1.2)[Second OPTIONS] return the value of
fetch(innerRequest)(rule 3.2.1.3)innerRequest's target IP address space is nowlocal, enter the internal logic (rule 2.1)connection's IP address space islocal, equal toinnerRequest's target IP address space, returnnull(rule 2.1.2)
Combining the Chromium code shown in the previous article, we can conclude that the last successful pre-flight was passed by the rule allowed-by-target-ip-address-space. So far, the reason why the request is retried is also clear.
I also understand a thing: no wonder the call chain went around and came back when tracing the code. It turned out that the IP address space is actually specified at runtime, and there is no scene in the code where "it is assigned to a constant".
There is still one last question: why is client_address_space loopback and resource_address_space unknown on my computer?
The Impact of Proxy on Address Types
I did not find a very clear condition for kUnknown in the specification (except that "the default value is kUnknown"), so I had to look at the source code again. In LocalNetworkAccessChecker::Check, I saw this:
mojom::IPAddressSpace resource_address_space =
TransportInfoToIPAddressSpace(transport_info);And the function called inside is written as follows:
IPAddressSpace TransportInfoToIPAddressSpace(const net::TransportInfo& info) {
switch (info.type) {
case net::TransportType::kDirect:
case net::TransportType::kCached:
return IPEndPointToIPAddressSpace(info.endpoint);
case net::TransportType::kProxied:
case net::TransportType::kCachedFromProxy:
return mojom::IPAddressSpace::kUnknown;
}
}That is to say, if it is kDirect (direct) or kCached (cached), the IP address space is calculated normally, but if it is kProxied (proxy) or kCachedFromProxy, then it is kUnknown. I turned my attention to the Switchy Omega plugin in the upper right corner of the browser...
Bingo, when I turn off the Switchy Omega plugin or set it to direct mode, the problem can be reproduced stably on my computer...
Summary
There are a lot of things written in this article. To sum up, my troubleshooting steps are:
- Check NetLog to find out that the reason may be that the PNA rules are violated;
- Understand the basic concept of PNA and find out the cause is intranet DNS;
- Trace the Chromium source code to find out the target IP address space;
- Read the specification to understand the calculation rules of target IP address space, and find out the reason for the repeated requests by combining the source code;
- Trace the Chromium source code to find out the impact of proxy plugins on IP address space.
During the process, I raised several questions, and they were all answered:
What is PNA?
- PNA (private network access, also known as local network access) is a security feature designed for browsers to prevent external websites from initiating intranet requests through browsers (without confirmation), which may cause the intranet with relatively weak protection to be attacked.
Why is the fetch request on my colleague's computer judged as an intranet request? Why does this violate the PNA rules?
- Because the intranet DNS configuration of the company's platform contains the domain of the fetch target, but not the domain of the page, so in the browser's view, a
localrequest is initiated on apublicpage, which will be intercepted by the PNA'sblocked-by-policy-preflight-warnrule at this stage.
- Because the intranet DNS configuration of the company's platform contains the domain of the fetch target, but not the domain of the page, so in the browser's view, a
Why is there a retry after the first pre-flight fails?
- Because the first pre-flight request returned
network error, according to the PNA specification, theHTTP-no-service-workerfetch process will be recursively called.
- Because the first pre-flight request returned
Why do I see the same parameters in the two pre-flight requests in NetLog, but the second one succeeded?
- The parameters to be verified in these two pre-flight requests are not exactly the same - except for
client_address_spaceandresource_address_space, there is also atarget_address_spacethat only appears in the code; - The
target_address_spaceof the second pre-flight request islocal(set after the first failure), while the first one isnull.
- The parameters to be verified in these two pre-flight requests are not exactly the same - except for
Why is
client_address_spaceloopbackandresource_address_spaceunknownon my computer?- Because I use a proxy plugin, it will cause Chrome to judge
resource_address_spaceasunknown.
- Because I use a proxy plugin, it will cause Chrome to judge
Update
Errors in the Issues Panel
As pointed out by a friend, PNA issues will be displayed in the issues panel:
Why didn't I see it at first? It may be because the Issues panel was mainly used to display some issues related to SameSite Cookie before, which I think are edge cases (cannot be solved in the project for now), so I just ignored it. It seems that in the future, when checking errors, we should look not only at the console panel, but at the issues panel as well.