求助大佬们一个很奇怪的PHP问题！已问遍所有主流ai

Telegramr

func 发表于 2025-2-27 17:13
服务器那边把curl结果print_r存到一个txt文件看看先

还是本地返回正确的数据

服务器上应该是返回空白的

胖虎_

curl -L http://yourwebsite.com  
百度给的命令，你试试看，他有几层跳转。
我没看你的代码不知道具体代码。

可以换个服务器试试，也可能这个IP被你要请求的网址拉黑了。

func

Telegramr 发表于 2025-2-27 18:00
还是本地返回正确的数据

服务器上应该是返回空白的

刚才测试了下你爬取的链接，那边有一层waf，应该是被识别成bot拦截了。
你要研究下怎么过掉这个东西，才能拿到数据。
你说你本地测试没问题，会不会是携带的cookie关联本地ip，你服务器要重新存一份cookie，当然这只是我的猜测，你自己研究一下

Telegramr

func 发表于 2025-2-28 08:55
刚才测试了下你爬取的链接，那边有一层waf，应该是被识别成bot拦截了。
你要研究下怎么过掉这个东西，才 ...

谢谢您的回复，本地怎么关联cookie呢？我没有进行过这个操作，但是我查看了网上的很多python的代码，他们都没有这个操作。

我刚才在服务器上使用命令行curl了一遍，仍然返回那个错误提示。

1. curl -L -X GET "https://www.cwl.gov.cn/cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5" \
   
2. > -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36" \
   
3. > -H "Referer: https://www.cwl.gov.cn/" \
   
4. > -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" \
   
5. > -H "Accept-Language: zh-CN,zh;q=0.9,en;q=0.8" \
   
6. > -H "Accept-Encoding: gzip, deflate, br, zstd" \
   
7. > -H "Cache-Control: max-age=0" \
   
8. > -H "Connection: keep-alive" \
   
9. > -H "Cookie: 21_vq=9; HMF_CI=96ce8230eb88a79f892aefc97605870f4c3e6a7826499fdd1a4663e39a878a352ab87872e84207f940421846dc3cad1ea727aef41e54c73a95c2ee89acf3a2a181"
   
10. curl: (47) Maximum (50) redirects followed
    

复制代码

Telegramr

@胖虎 不知道为什么我回复不了您，发布没有任何反应。

我试过了curl，仍然是返回那个错误提示，应该和IP没有多大关系，因为我代码是在本地写完并调试后才上传到服务器的。一上传就提示。之前没有进行过这样的操作。

imdong

第一次访问的时候 服务器做了302跳转，并设置了cookie  后面请求服务器会判断改cookie，
第一步头信息设置UA和Referer 发送请求
第二步 判断返回状态码是否为302
第三步：如果状态码时302 则获取返回头中的cookie、和location
第四步：请求第三部拿到的location 并在第一步请求的头信息的基础上设置第三部拿到的cookie
就可以拿到正确的json字符串了

Telegramr

imdong 发表于 2025-2-28 10:03
第一次访问的时候 服务器做了302跳转，并设置了cookie  后面请求服务器会判断改cookie，
第一步头信息设置U ...

我在服务器上使用命令行curl操作，返回以下信息

1. curl -L -v --max-redirs 100 -X GET "https://www.cwl.gov.cn/cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC" \
   
2. -H "User-Agent: baidu：Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" \
   
3. -H "Referer: https://www.cwl.gov.cn/" \
   
4. -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" \
   
5. -H "Accept-Language: zh-CN,zh;q=0.9,en;q=0.8" \
   
6. -H "Accept-Encoding: gzip, deflate, br, zstd" \
   
7. -H "Cache-Control: max-age=0" \
   
8. -H "Connection: keep-alive" \
   
9. -H "Cookie: 21_vq=9; HMF_CI=96ce8230eb88a79f892aefc97605870f09846e72de625cb216052159c751b0656906343280a05f6ecd334d49c2f5bce058e4a38f6125c41d6ad49e5d8b6163c9a7"

复制代码

< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Fri, 28 Feb 2025 02:11:34 GMT
< Content-Type: text/html
< Content-Length: 138
< Connection: keep-alive
< Cache-Control: no-store
< Location: https://www.cwl.gov.cn/cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC
< x-ws-request-id: 67c11b56_PS-JJN-01wa2247_64743-919
< Set-Cookie: HMF_CI=96ce8230eb88a79f892aefc97605870fc4779740f71f0e749300291daecfcc32c9ea6cfe7736aa2cf993ef4782f7f87b1f448c009101c148709fbd1ad72551597f; Expires=Sun, 30-Mar-25 02:11:34 GMT; Path=/
<
* Ignoring the response-body
* Connection #0 to host www.cwl.gov.cn left intact
* Issue another request to this URL: 'https://www.cwl.gov.cn/cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC'
* Re-using existing connection! (#0) with host www.cwl.gov.cn
* Connected to www.cwl.gov.cn (27.148.164.154) port 443 (#0)
> GET /cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC HTTP/1.1
> Host: www.cwl.gov.cn
> User-Agent: baidu：Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
> Referer: https://www.cwl.gov.cn/
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
> Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
> Accept-Encoding: gzip, deflate, br, zstd
> Cache-Control: max-age=0
> Connection: keep-alive
> Cookie: 21_vq=9; HMF_CI=96ce8230eb88a79f892aefc97605870f09846e72de625cb216052159c751b0656906343280a05f6ecd334d49c2f5bce058e4a38f6125c41d6ad49e5d8b6163c9a7
>
< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Fri, 28 Feb 2025 02:11:34 GMT
< Content-Type: text/html
< Content-Length: 138
< Connection: keep-alive
< Cache-Control: no-store
< Location: https://www.cwl.gov.cn/cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC
< x-ws-request-id: 67c11b56_PS-JJN-01wa2247_64743-920
< Set-Cookie: HMF_CI=96ce8230eb88a79f892aefc97605870fc4779740f71f0e749300291daecfcc63c9ea6cfe7736aa2cf993ef4782f7f87b1f448c009101c148709fbd1ad72551597f; Expires=Sun, 30-Mar-25 02:11:34 GMT; Path=/
<
* Ignoring the response-body
* Connection #0 to host www.cwl.gov.cn left intact
* Issue another request to this URL: 'https://www.cwl.gov.cn/cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC'
* Re-using existing connection! (#0) with host www.cwl.gov.cn
* Connected to www.cwl.gov.cn (27.148.164.154) port 443 (#0)
> GET /cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC HTTP/1.1
> Host: www.cwl.gov.cn
> User-Agent: baidu：Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
> Referer: https://www.cwl.gov.cn/
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
> Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
> Accept-Encoding: gzip, deflate, br, zstd
> Cache-Control: max-age=0
> Connection: keep-alive
> Cookie: 21_vq=9; HMF_CI=96ce8230eb88a79f892aefc97605870f09846e72de625cb216052159c751b0656906343280a05f6ecd334d49c2f5bce058e4a38f6125c41d6ad49e5d8b6163c9a7
>
< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Fri, 28 Feb 2025 02:11:34 GMT
< Content-Type: text/html
< Content-Length: 138
< Connection: keep-alive
< Cache-Control: no-store
< Location: https://www.cwl.gov.cn/cwl_admin/front/cwlkj/search/kjxx/findDrawNotice?name=ssq&issueCount=5&pageNo=1&pageSize=5&systemType=PC
< x-ws-request-id: 67c11b56_PS-JJN-01wa2247_64743-921
< Set-Cookie: HMF_CI=96ce8230eb88a79f892aefc97605870fc4779740f71f0e749300291daecfcc39c9ea6cfe7736aa2cf993ef4782f7f87b1f448c009101c148709fbd1ad72551597f; Expires=Sun, 30-Mar-25 02:11:34 GMT; Path=/
<
* Ignoring the response-body
* Connection #0 to host www.cwl.gov.cn left intact
* Maximum (100) redirects followed
curl: (47) Maximum (100) redirects followed
* Closing connection #0

imdong

Telegramr 发表于 2025-2-28 10:20
我在服务器上使用命令行curl操作，返回以下信息

服务器做了302 所以要获取302设置的cookie 然后重新请求

imdong

代码

结果


代码随意写的  自己优化一下

Telegramr

imdong 发表于 2025-2-28 10:23
代码

结果

谢谢您的指教，完美解决