[]

PHPCE,PHP-Coroutine-Engine

php7php7

phpgithubforkphp7.1.17

golangopenrestyjavaswoole

rpcmysqlredis

IO

php-fpm

php-fpmphp-fpmnginxCPUphp

email :[email protected] ,

WorkerManworker,swoole

(DNS)

IOphp-fpmPHP

PHP-FPM

phpphp

phpphp()TSRM

TSRMphpphp

EG()SG()emallocefree

php-fpm --enable-maintainer-zts zts

phpphp

php(TSRM)php-fpmphp

2016MAC PRO28G

wrk,

./wrk -c 800 -t 800 -d 120s http://localhost/test.php

800 800800

2400qps

:

wrk >> nginx >> php-fpm >> nodejs

wrkNGINX FASTCGInginxphp-fpm, php-fpmNODEJS

nodejs80802

test.js: tutorial/test.js

nodejs :

 node tutorial/test.js

test.php: tutorial/test.php

test.php coro_http_get(http)test.js

 var_dump(coro_http_get("http://localhost:8080/"));

php-fpm 8PHP-FPM128M

NGINX8

1

2

curl ,

82Qps 1858002Qps400

php-fpmphp-fpm128php-fpmphp-fpmqps642fpm2

taskcpu

TOPkernel_task43%LINUX

kernel_task wrk nginx php-fpm nodejs 4

php-fpmcpu

NODEJS18%WRK10%

nginx cpuphp-fpmphp-fpmnginx

1.php-fpmphp-fpm128.

rootPHP_COROUTINE_ENGINE_COUNT

echo 'export PHP_COROUTINE_ENGINE_COUNT=1024' >> ~/.bash_profile
source ~/.bash_profile

/etc/sudoerssudo

Defaults    env_reset 
 
Defaults    !env_reset

2.PHP-FPM,php-fpm.d/www.conf

 pm = static  
 pm.max_children = 4

3.,.php.ini

 memory_limit = 128M

4.chromenginxcurl

ext/coro_http PHPcoro_http_get coro_http.c, demo,demo php-fpm

macOSlinux

:

1.libevent,php

2.automake

 sh buildconf --force

3.php

 ./configure --prefix=/usr/local/php7 --enable-fpm --enable-coro_http --enable-maintainer-zts && make && sudo make install

linux --with-openssl mac

MACiconvmakeMakefile-liconv

4.php-fpmPHP-FPMCPU1php-fpm.d/www.conf

 pm = static   
 pm.max_children = 1

5.php-fpm

 sudo /usr/local/php7/sbin/php-fpm

6.nginx,nginxtutorialtest.php,

7.NGINXmacchromeurlcurl chrome, http://localhost/test.php?a=xx chrome

Docker hub[]:

docker pull phpce/php-coroutine-engine  
docker run --privileged --cpus=2 -e 'PHP_COROUTINE_ENGINE_COUNT=128' -p 8083:80 -v `pwd`:/data/www phpce/php-coroutine-engine

 http://localhost:8083/test3.php

Docker build:

1.:(make ,make dist clean)

 docker build -t php-coroutine-engine -f tutorial/Dockerfile ./

2.docker:

 docker run --privileged --cpus=2 -e 'PHP_COROUTINE_ENGINE_COUNT=128' -p 8083:80 -v `pwd`/tutorial:/data/www php-coroutine-engine

 http://localhost:8083/test3.php

"window.baidu.sug({\"q\":\"\",\"p\":false,\"bs\":\"\",\"csor\":\"0\",\"status\":769,\"s\":[]});"

test3.php

<?php
echo json_encode(coro_http_get("http://sp0.baidu.com/5a1Fazu8AA54nxGko9WTAnF6hhy/su"));
?>

---

Project description[For English]

Project name phpce,full name php-coroutine-engine

This project is a branch of php7 and has implemented PHP Native Coroutine.

This project is a version from PHP's official GitHub fork, based on php7.1.17 version.

Coopera is a design pattern that supports high concurrent servers.

Now the mainstream server-side languages and frameworks support the call of the association Coroutine, including golang, openresty, Java, swoole and so on.

Coroutine process can reduce server congestion, and the use of Coroutine process can significantly improve server performance for services such as RPC, mysql, redis, and other services that need to use remote calls.

The nature of association Coroutine is one of the implementations of asynchronous asynchronous non blocking IO model.

The traditional php-fpm design mode is mainly through concurrent processing requests through multiple processes. The use of server resources is inadequate.

This project is through the transformation of the php-fpm source code, the implementation of the Coroutine model of the php-fpm, and finally, like nginx, several processes can deal with a large number of concurrent requests, full use of CPU resources, multiplied the performance of the PHP server.

Note: this item has not been completely tested. Please be careful not to use it in the production environment.

Author email :[email protected] ,if you have question ,send message to me.

Finally, it's not easy to finish the project. Thank you for the support of the project and colleagues, especially the WorkerMan author worker, Chen Lei from didi, tongge who is the core developer of swoole, and my direct leader, hongjie. Next, we need to test in real applications and develop more extensions that support coroutine. I also hope that more developers will join in and give support and guidance.

Coroutine theory

The essence of the Coroutine is one of the asynchronous non blocking IO implementations. If php-fpm wants to implement the syndication, it needs to be divided into two parts, one is the scheduler, the other is to implement the syndication logic in the PHP extension.

PHP-FPM is the role of the scheduler.

When a request comes in, the scheduler triggers one execution. During the execution of this request, if a remote call is encountered in the PHP code, PHP automatically gives control to the scheduler after the request is sent. At this point, the scheduler can decide whether to handle other new incoming requests or to process requests that can be executed after the previous remote call.

Each request is bound from a Coroutine process context from the Coroutine context pool, and after the request is finished, the coroutine process context is put back so that other requests can be reused.

The context of the association mainly stores the global and private variables used in the PHP running environment. This implementation is mainly based on PHP's multithread secure storage (note not multithreading) and TSRM related library.

The TSRM library will isolate all PHP global variables from macros and PHP memory pools.

So when developing the extension of the coroutine, we should pay attention to the use of macro variables such as EG (), SG (), emalloc, EFREE and so on.

The --enable-maintainer-zts parameter tells the compiler that it is required to support ZTS multithreaded secure storage.

In multithread secure storage mode, the way of storing internal memory used by PHP can be generally understood as a matrix. The longitudinal axis of a matrix is each thread (coroutine). The horizontal axis of the matrix stores all the memory needed in all the running environments. That is to say, each PHP association has its own memory.

The PHP version of the coda version uses multithread secure storage (TSRM). So the expansion of this version of php-fpm and the PHP code of application layer do not support multithreading.

Pressure test report

The system has not been optimized. It will be optimized in the future

The machine is 2016 MAC PRO, and the configuration is 2 cores, 8G memory.

The test tool is wrk, and the command is as follows

./wrk -c 800 -t 800 -d 120s http://localhost/test.php

Open 800 threads, 800 connection presses, equivalent to 800 concurrent requests.

In theory, each request is returned in 2 seconds, and the limit test result is 400qps.

The request process is:

wrk >> nginx >> php-fpm >> nodejs

The wrk client accesses the NGINX FASTCGI reverse proxy, nginx visits php-fpm, and php-fpm visits the NODEJS program through the coroutine function.

The nodejs program is on the 8080 port, simulating the interface that returns the result in 2 seconds.

Test.js program location: tutorial/test.js

The nodejs boot command:

Node tutorial/test.js

Test.php location: tutorial/test.php

Test.php calls test.js through coro_http_get (HTTP client to implement the Association).

var_dump (coro_http_get ("http://localhost:8080/"));

In this test, php-fpm started 8 processes, and the maximum memory usage per process in PHP-FPM is 128M.

NGINX started 8 processes

The results of the test are as follows:

System occupancy capture 1:

System occupancy capture 2:

In the process of pressure measurement, access through curl can be accessed normally.

Summary:

The overall test results, although the number of processes is only 8, and the remote interface 2 seconds to return. Qps is 185. Because it is 800 concurrent, each request for 2 seconds, the limit Qps should be 400.

The result is much lower than I thought. It is estimated that it needs to be optimized. But from this measurement result, we can see that the difference between the coroutine and traditional php-fpm is still very different. Traditional php-fpm usually needs to open many more processes, and servers usually open 128 processes, even ignoring the impact of php-fpm multi-processes on efficiency. Then the maximum QPS theory that traditional php-fpm can provide is no more than 64 (because each request returns in 2 seconds, and the FPM process blocks for 2 seconds).

The result of pressure measurement is more time-out. This is related to the CPU occupied by the system task

Analyze the resource occupation of the TOP command, for example, the first resource map. Kernel_task takes up 43% of the resources, which is quite high. I don't know if the performance below LINUX will be better

The kernel_task is higher, it should be wrk nginx php-fpm nodejs that the network transmission scheduling of these four groups of programs is done by it in the whole process of pressure measurement. It is busy.

The CPU usage of php-fpm is relatively low, which means that the scheduling is handed over to the kernel for execution. This conforms to the characteristics of the co process.

In the figure, NODEJS takes about 18% and WRK takes up 10%

Nginx occupies CPU slightly below php-fpm. In another way, the performance of php-fpm is nearly nginx.

Finally, because the machine presses the machine, there are more services. If separated, the effect would be better.

Matters of attention

1. Keep this in mind. In php-fpm, the coroutine pool size of each php-fpm process is fixed to 128 by default. If you want the machine to have better concurrency, please configure the process number reasonably. you can set the coroutine number in environmental variables.

Modify the coroutine pool size in each process:

First,Checkout to root user,then add environmental variables in environmental variables(PHP_COROUTINE_ENGINE_COUNT is couroutine number in every process)

echo 'export PHP_COROUTINE_ENGINE_COUNT=1024' >> ~/.bash_profile

source ~/.bash_profile

Modify configuration in /etc/sudoers to support sudo's environment variable delivery:

Defaults env_reset

For:

Defaults! Env_reset

2. when starting, pay attention to adjusting the number of PHP-FPM processes, php-fpm.d/www.conf

pm = static  
pm.max_children = 4

3. note that adjust the memory size used by each process. In php.ini. Please calculate reasonably according to the number of processes and the available memory of the system.

memory_limit = 128M

4. when testing, please note that some browsers are blocking the same link, but changing the parameters will not block, such as chrome (this comparison pit, at first I think it's a long time to think it's nginx's problem). It is recommended that the curl command test be used to better observe the synergy effect.

Extension explanation

The ext/coro_http directory provides a coro_http_get method for testing the PHP extension of the co development.

The core document is coro_http.c, which contains detailed annotations.

This extension can be thought of as a demo for the implementation of a collaboration, which is very simple, and developers can develop more extensions to support the collaboration with the help of demo.

Because the part of scheduler in php-fpm has been implemented, developers only need to extend it to develop cooperating applications.

Only macOS and Linux are currently supported

Debug method 1, compile and install:

1. Install libevent libraries in the system first. Find out how to install libevent libraries by yourself, and you may also need to install some extensions required by PHP

2. generate automake configuration file

sh buildconf --force

3. install PHP

./configure --prefix=/usr/local/php7 --enable-fpm --enable-coro_http --enable-maintainer-zts && make && sudo make install

Note: --with-openssl is needed in Linux, otherwise installation will be wrong and Mac will not be needed.

4. after installation, modify the php-fpm configuration file and set the number of PHP-FPM processes. The online environment can be adjusted to 1 (php-fpm.d/www.conf) according to the number of CPU and the number of required cooperations.

This is mainly the two parameters

pm = static  
pm.max_children = 1

5. start php-fpm

sudo /usr/local/php7/sbin/php-fpm

6. Configuration of nginx, please consult the relevant information, please configure the nginx access directory into the source code tutorial directory, mainly the test. php, for testing

7. You can start the test without completing it. According to the configured NGINX, access the browser directly (Note: the chrome in MAC does not support the same URL concurrent access. It is recommended to use the curl command test)

If you strongly request the use of chrome, please read the following text carefully.

For example, http://localhost/test.php?a=xx

It is important to note that in chrome, two windows can be accessed, but the following parameters are different.

Debug method twoDocker hub installation[recommended]:

1.pull images from docker hub

docker pull phpce/php-coroutine-engine  
docker run --privileged --cpus=2 -e 'PHP_COROUTINE_ENGINE_COUNT=128' -p 8083:80  -v `pwd`:/data/www phpce/php-coroutine-engine

2.browsers enter the URL

 http://localhost:8083/test3.php

Debug method three, Docker build installation:

1. enter root dir of project, execute:(if you had make in this dir please run make dist clean)

 docker build -t php-coroutine-engine -f tutorial/Dockerfile ./

2. run docker:

docker run --privileged --cpus=2 -e 'PHP_COROUTINE_ENGINE_COUNT=128' -p 8083:80 -v `pwd`/tutorial:/data/www php-coroutine-engine

3. browsers enter the URL

http://localhost:8083/test3.php

It can be seen that the output results are as follows:

"window.baidu.sug({\"q\":\"\",\"p\":false,\"bs\":\"\",\"csor\":\"0\",\"status\":769,\"s\":[]});"

Test3.php Code:

<?php
echo json_encode(coro_http_get("http://sp0.baidu.com/5a1Fazu8AA54nxGko9WTAnF6hhy/su"));
?>