解决NGINX+PHP-FPM failed to ptrace(PEEKDATA) Input/output error出错问题

23113fff

网站总是出现bad gateway 提示，时有，时无，查看了一下日志，居然出现一堆错误，如下

[29-Mar-2014 22:40:10] ERROR: failed to ptrace(PEEKDATA) pid 4276: Input/output error (5)

[29-Mar-2014 22:53:54] ERROR: failed to ptrace(PEEKDATA) pid 4319: Input/output error (5)

[29-Mar-2014 22:56:30] ERROR: failed to ptrace(PEEKDATA) pid 4342: Input/output error (5)

[29-Mar-2014 22:56:34] ERROR: failed to ptrace(PEEKDATA) pid 4321: Input/output error (5)

[29-Mar-2014 22:56:40] ERROR: failed to ptrace(PEEKDATA) pid 4314: Input/output error (5)

网上也找了很多方法，很多人说是rlimit_files 打开文件数的问题，但是觉得不太靠谱，最后找到鬼佬的话，看上去还有几分道理。

http://serverfault.com/questions/406532/i-o-error-with-php5-fpm-ptracepeekdata-failed

It appears you have request_slowlog_timeoutenabled. This normally takes any request longer than N seconds, logs that it was taking a long time, then logs a stack trace of the script so you can see what it was doing that was taking so long.

In your case, the stack trace (to determine what the script is doing) is failing. If you’re running out of processes, it is because either:

• After php-fpm stops the process to trace it, the process fails to resume because of the error tracing it
• The process is resuming but continues to run forever.
  
  

My first guess would be to disable request_slowlog_timeout. Since it’s not working right, it may be doing more harm than good. If this doesn’t fix the issue of running out of processes, then set the php.ini max_execution_time to something that will kill the script for sure.

看样子是因为我打开了slowlog  然后，再设置 了 request_slowlog_timeout 这个参数，，所以后php 没有执行完就出错了。。

上面解决的办法是：

禁用 php-fpm.conf 里的  request_slowlog_timeout  和 slowlog  ，然后，修改 php.ini 里的 max_execution_time 参数

下面为英文版, 说的更详细,大家看看.

If you're running PHP-FPM you can see these kind of errors in your PHP-FPM logs.

$ tail -f php-fpm.log[29-Dec-2015 23:03:10] NOTICE: child 11667 stopped for tracing[29-Dec-2015 23:03:10] NOTICE: about to trace 11667[29-Dec-2015 23:03:10] ERROR: failed to ptrace(PEEKDATA) pid 11667: Input/output error (5)[29-Dec-2015 23:03:10] NOTICE: finished trace of 11667

So, what do they mean?

They are the result of a configuration directive and a flaw in the way PHP-FPM handles requests.

Slowlog stacktraces

If your PHP-FPM configuration contains the request-slowlog-timeout parameter, the PHP-FPM master process will attempt to get a stacktrace of the running process when it exceeds that timeout.

It's a great way to identify 'slow' processes and peek inside the request, to see what happened at that time. It mostly works, too.

On busy servers however, you can see the logs start to fill like this.

[29-Dec-2015 23:03:10] ERROR: failed to ptrace(PEEKDATA) pid 11667: Input/output error (5)[29-Dec-2015 23:04:10] ERROR: failed to ptrace(PEEKDATA) pid 11668: Input/output error (5)[29-Dec-2015 23:05:12] ERROR: failed to ptrace(PEEKDATA) pid 11668: Input/output error (5)...Avoiding these errors in the logs

First, let me start by saying it's a cosmetic issue: PHP itself isn't having problems. It's not even affected by this. But it can't log the stacktrace that was requested, so it reports this error.

If this bothers you, you can disable it by commenting the following 2 parameters in your PHP-FPM logs:

;slowlog = /var/log/php-fpm/slow.log;request_slowlog_timeout = 5s

(you comment them by adding a semicolon in front of the line and restarting your PHP-FPM daemon)

That will make the errors disappear. It also stops any kind of slowlogging from ever happening.

What's causing the PEEKDATA error?

I already mentioned it's a result of a flaw in how PHP handles requests. This is explained in more detail here . The gist of it is:

... the worker is free to go when the master is determining slow execution. When stopping to be traced, it may have completed that execution and is in any stage serving another request, so the tracer gets the chance of failure or worse, dumping out the stack of an irrelevant execution.

FPM slow log sucks

In other words: when PHP's slowlog tries to get an actual stacktrace, it may send a signal to the master process which in turn finds the child process that is slow (which itfork() 'd). However, by the time that happens, the child may have already finished its request and started serving another one.

Or it's not even handling a request at all, because it's finished and waiting for a new incoming request.

This leads to A) a stacktrace of the wrong process or B) the error you see above, a PEEKDATA error because there was no data to peek into.

So take that into account when reviewing PHP FPM slowlogs, too.