Compiling PHP

Compiling PHP 7.1 on Ubuntu 14.04 x86_64

After going thru all of troubles with compiling PHP, which was actually a fun process that took around 4 days in total, i wanted to share the how-to guide, for myself to remember and for other to have a reference. This guide is not for beginners, so i will keep the story short by providing minimum guidance.

There are many different reasons why u would want to compile PHP from sources i.e.

Curious how it works
Your distribution doesnt support needed version, neither there 3rd party distro available
Getting the last bits of performance by compiling it your way
Beta testing against latest PHP features
Building your own/company’s distribution, because everyone has their special requirements
Improving your software CI/CD
Learning new stuff altogether

The goal

Our goal is to have our own stable redistributable package (deb) of custom PHP build, the steps would be

Read docs
Meet dependencies
Fetch PHP
Compile PHP
Compile/phpize extension
Generate INI files
Build deb package
Benchmarking

Read docs

The is alot of googling involved in the process, and to make life easier i references all the info at the end of article.

Meet dependencies

Grab a Ubuntu 64 distro, and lets begin

update-alternatives --set editor /usr/bin/vim.basic

apt-get -y install \
libt1-dev \
libgmp-dev \
libcurl4-openssl-dev \
bison \
libxslt-dev \
libxml2-dev \
libxpm-dev \
libmcrypt-dev \
pkg-config \
libbz2-dev \
libpng-dev \
libfreetype6-dev \
libgmp3-dev \
libmysqlclient-dev \
libwebp-dev \
libjpeg-dev \
build-essential \
libtool \
software-properties-common \
libssl-dev \
autoconf \
git-core \
zlib1g-dev \
libc-bin \
cmake

apt-get install pkg-config

add-apt-repository ppa:ubuntu-toolchain-r/test -y
apt-get update
apt-get install gcc-4.8 g++-4.8 -y
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.8

This installs my set of required packages for building [1].

If u need more, for example you need webp support for GD extension for imagewebp function then according to GD documentation you will need to install

As of PHP 7.0.0 --with-webp-dir=DIR has to be added
apt-get install libwebp-dev
--with-webp-dir=/usr

And so on for other extensions not included in this guide. Second, we will use gcc 4.8 according to this gcc roadman, its fresh enough.

Fetch PHP

Lets compile latest released version atm. 7.0.12

To make it more challenging and be real-world case, lets compile it with some extensions

with one static extension redis
with one shared extension via phpize amqp
and our amqp extension has another dependency rabbitmq-c or librabbitmq-dev

cd ~

PHP_VERSION="7.0.12"
PHP_FULLVERSION="php-${PHP_VERSION}"
PHP_SOURCES_FILE="${PHP_FULLVERSION}.tar.gz"
PHP_SOURCES_LOCATION="http://de1.php.net/get/${PHP_SOURCES_FILE}/from/this/mirror"
PHP_EXT_REDIS_VERSION="3.0.0"

wget -O $PHP_SOURCES_FILE $PHP_SOURCES_LOCATION
tar zxf $PHP_SOURCES_FILE
cd $PHP_FULLVERSION

cd ext
wget https://pecl.php.net/get/redis-${PHP_EXT_REDIS_VERSION}.tgz
tar zxf redis-${PHP_EXT_REDIS_VERSION}.tgz
mv redis-${PHP_EXT_REDIS_VERSION} redis
cd ..

Compile PHP

Here we need to make alot of decisions, here is the end-result

PHP_ZEND_API="20151012"
PHP_INSTALL_ROOT="/usr/local/php7"
PHP_PREFIX_DIR="${PHP_INSTALL_ROOT}/${PHP_VERSION}"
PHP_DATAROOT="${PHP_PREFIX_DIR}/share"
PHP_LIB_DIR="${PHP_PREFIX_DIR}/lib"
PHP_BIN_DIR="${PHP_PREFIX_DIR}/bin"
PHP_EXTENSIONS_DIR="${PHP_LIB_DIR}/${PHP_ZEND_API}"

PHP_INITD_DIR="${PHP_PREFIX_DIR}/init.d"
PHP_INI_CLI_DIR="${PHP_PREFIX_DIR}/etc"
PHP_INI_CLI_FILE="${PHP_INI_CLI_DIR}/php.ini"
PHP_INI_CLI_SCAN_DIR="${PHP_INI_CLI_DIR}/conf.d"
PHP_INI_CLI_EXTENSIONS="${PHP_INI_CLI_SCAN_DIR}/extension.ini"
PHP_INI_CLI_ZENDEXTENSIONS="${PHP_INI_CLI_SCAN_DIR}/zend_extension.ini"

PHP_INI_FPM_DIR="${PHP_INI_CLI_DIR}"
PHP_INI_FPM_FILE="${PHP_INI_FPM_DIR}/php-fpm.conf"
PHP_INI_FPM_SCAN_DIR="${PHP_INI_FPM_DIR}/php-fpm.d"

PHP_AMQP_RABBITMQ_LIBDIR="${PHP_LIB_DIR}/rabbitmq-shared"

export LANG="en"
export LC_ALL="en_US.UTF-8"
export CHOST="x86_64-linux-gnu"

SAFE_CFLAGS_1="-m64 -march=corei7 -mfpmath=sse -minline-all-stringops -pipe -fstack-protector -Wformat -Werror=format-security"
SAFE_CFLAGS_2="-fPIC -fPIE -fno-strict-aliasing -fsigned-char -std=gnu99 -mpc64 --param=ssp-buffer-size=4"

CFLAGS="\"-O3 ${SAFE_CFLAGS_1} ${SAFE_CFLAGS_2} "\"

CXXFLAGS=$CFLAGS

CMAKE_CFLAGS="-m64 -march=corei7 -O2 -fPIC"

We are building

for intel 64bit cpu with -march=corei7 [19] gcc flag
for (debian amd64) or (gnu x86_64) which are the same (see cat /usr/share/dpkg/cputable)
everything is installed under /usr/local/php7
we generate code for a 64-bit environment only [19]
we enable some optimizations like -mfpmath=sse -minline-all-stringops (-mfpmath=sse is default for x86-64 compiler [19])
-fstack-protector --param=ssp-buffer-size=4 is hardening option [30] for GCC<4.9, that emit extra code to check for buffer overflows
-fno-strict-aliasing is performance option passed through to the link stage (34) (35), disables -O2 and -O3 -fstrict-aliasing (36)
-std=gnu99 enables C99 standard (23) (25), i.e. C code already contains long long int data types; also C99 brings inline functions and optimizations
-fPIC -fPIE is similar to -fpic -fpie (37) (38) Generate position-independent code (PIC) & executable
-fsigned-char Let the type char be signed, like signed char. (22)
-mpc64 rounds the significands of results of floating-point operations to 53 bits (double precision) vs 64bits (default)

We also define CMAKE_CFLAGS flags for later on.

Now as we have PHP sources, lets deal our extension dependency librabbitmq. First we need to check if we dont have it installed pkg-config librabbitmq --libs. Because (26) GCC first searches for libraries in /usr/local/lib, then in /usr/lib. Following that, it searches for libraries in the directories specified by the -L parameter, in the order specified on the command line. If we do have librabbitmq library installed, GCC will link agains it ignoring our library.

#REMOVE ANY PREINSTALLED PACKAGE (check `pkg-config librabbitmq --libs`)

RABBITMQ_C_VERSION="0.8.0"
RABBITMQ_C_SOURCES_LOCATION="https://github.com/alanxz/rabbitmq-c/archive/v${RABBITMQ_C_VERSION}.tar.gz"

rm -rf rabbitmq-c
wget -O rabbitmq-c.tar.gz ${RABBITMQ_C_SOURCES_LOCATION}
tar zxf rabbitmq-c.tar.gz
rm -f rabbitmq-c.tar.gz
mv rabbitmq-c-${RABBITMQ_C_VERSION}/ rabbitmq-c
cd rabbitmq-c
rm -rf build && mkdir build && cd build

rm -f CMakeCache.txt

cmake -DCMAKE_C_FLAGS="$CMAKE_CFLAGS" \
-DCMAKE_INSTALL_PREFIX=${PHP_AMQP_RABBITMQ_LIBDIR} \
-DBUILD_EXAMPLES=OFF \
-DBUILD_STATIC_LIBS=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_TOOLS=OFF \
-DENABLE_SSL_SUPPORT=OFF \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_API_DOCS=OFF ..

mkdir -p ${PHP_LIB_DIR}
cmake --build . --target install --clean-first

cd ../..

This will install our librabbitmq under our PHP root for linking later on.

rm -f configure && ./buildconf --force

CONFIGURE_STRING=" "\
"--prefix=${PHP_PREFIX_DIR} "\
"--bindir=${PHP_BIN_DIR} "\
"--sbindir=${PHP_PREFIX_DIR}/sbin "\
"--sysconfdir=${PHP_PREFIX_DIR}/etc "\
"--sharedstatedir=${PHP_PREFIX_DIR}/com "\
"--localstatedir=${PHP_PREFIX_DIR}/var "\
"--libdir=${PHP_LIB_DIR} "\
"--includedir=${PHP_PREFIX_DIR}/include "\
"--datarootdir=${PHP_DATAROOT} "\
"--infodir=${PHP_DATAROOT}/info "\
"--localedir=${PHP_DATAROOT}/locale "\
"--mandir=${PHP_DATAROOT}/man "\
"--docdir=${PHP_DATAROOT}/doc "\
"--with-config-file-path=${PHP_INI_CLI_DIR} "\
"--with-config-file-scan-dir=${PHP_INI_CLI_SCAN_DIR} "\
"--disable-all "\
"--without-pear "\
"--enable-cli "\
"--disable-cgi "\
"--disable-phpdbg "\
"--disable-debug "\
"--disable-rpath "\
"--with-layout=GNU "\
"--enable-fpm "\
"--enable-pdo "\
"--with-mysql-sock=/var/run/mysqld/mysqld.sock "\
"--with-mysqli=mysqlnd "\
"--with-pdo-mysql=mysqlnd "\
"--enable-mysqlnd "\
"--with-pic "\
"--with-pcre-regex "\
"--with-jpeg-dir=/usr "\
"--with-png-dir=/usr "\
"--with-xpm-dir=/usr "\
"--with-freetype-dir=/usr "\
"--enable-gd-native-ttf "\
"--enable-gd-jis-conv "\
"--disable-static "\
"--with-readline "\
"--with-fpm-user=www-data "\
"--with-fpm-group=www-data "\
"--enable-dom=shared "\
"--enable-mbstring=shared "\
"--enable-redis=shared "\
"--enable-opcache=shared "\
"--with-mhash=/usr "\
"--enable-phar=shared "\
"--enable-fileinfo=shared "\
"--with-gd=shared "\
"--enable-session "\
"--enable-hash "\
"--enable-json "\
"--enable-filter "\
"--enable-libxml "\
"--enable-ctype "\
"--enable-tokenizer "\
"--enable-pcntl "\
"--enable-posix "\
"--enable-xml "\
"--enable-xmlwriter "\
"--enable-bcmath "\
"--enable-simplexml "\
"--enable-exif "\
"--enable-shared "\
"--with-iconv "\
"--with-zlib "\
"--with-zlib-dir=/usr "\
"--with-libedit=/usr "\
"--with-curl "\
"--with-openssl=yes "\
"--with-pdo_mysql "\
"--build ${CHOST} "\
"--host ${CHOST} "\
"CC=gcc CFLAGS=$CFLAGS CHOST=$CHOST CXXFLAGS=$CFLAGS"

CFG_CMD="./configure ${CONFIGURE_STRING}"

eval $CFG_CMD;

make -j `cat /proc/cpuinfo | grep processor | wc -l`

make install

\cp php.ini-production ${PHP_INI_CLI_FILE}

Few notes on configure flags

readline is required for correct cli work
xml is required for utf8_decode & utf8_encode
most of the libraries are compiled-in, while only few are built as shared (i.e. --with-gd=shared)

What extensions you wish to build as shared is up to you, here is a mine sizes reference table

##AVG Shared Extension sizes##
# 3.8M  sqlite3.so
# 3.5M  fileinfo.so
# 2.8M  mbstring.so
# 1.7M  redis.so
# 1.3M  gd.so
# 1.1M  phar.so
# 989K  dom.so
# 923K  opcache.so
# 835K  zip.so
# 525K  openssl.so
# 436K  amqp.so
# 368K  sockets.so
# 358K  bcmath.so
# 300K  curl.so
# 246K  gmp.so
# 224K  simplexml.so
# 193K  exif.so
# 186K  zlib.so
# 173K  iconv.so
# 139K  pdo_sqlite.so
# 122K  xmlwriter.so
# 110K  xmlreader.so
#  99K  bz2.so
#  98K  posix.so
#  95K  pcntl.so
#  72K  tokenizer.so
#  50K  ctype.so

In the end we get our php installed under PHP_INSTALL_ROOT.

$ /usr/local/php7/7.0.12/bin/php -n --ini
Configuration File (php.ini) Path: /usr/local/php7/7.0.12/etc
Loaded Configuration File:         (none)
Scan for additional .ini files in: (none)
Additional .ini files parsed:      (none)

$ /usr/local/php7/7.0.12/sbin/php-fpm -v
PHP 7.0.12 (fpm-fcgi) (built: Oct 13 2016 11:38:14)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.12, Copyright (c) 1999-2016, by Zend Technologies

$ /usr/local/php7/7.0.12/bin/php -n -m
[PHP Modules]
bcmath
Core
ctype
curl
date
exif
filter
hash
iconv
json
libxml
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
posix
readline
Reflection
session
SimpleXML
SPL
standard
tokenizer
xml
xmlwriter
zlib

[Zend Modules]

Compile/phpize extension

Now lets build amqp extension, we already built librabbitmq-dev - An AMQP client library written in C - Dev Files

AMQP_VERSION="1.7.1"

wget https://pecl.php.net/get/amqp-${AMQP_VERSION}.tgz
rm -rf amqp
tar zxf amqp-${AMQP_VERSION}.tgz
mv amqp-${AMQP_VERSION} amqp
cd amqp

export PKG_CONFIG_PATH="${PHP_AMQP_RABBITMQ_LIBDIR}/lib/${CHOST}/pkgconfig/"
export LD_LIBRARY_PATH="${PHP_AMQP_RABBITMQ_LIBDIR}/lib/"

/usr/bin/pkg-config librabbitmq --libs

make clean && rm -f configure
${PHP_BIN_DIR}/phpize .
CFG_CMD="./configure \
--with-php-config=${PHP_BIN_DIR}/php-config \
--with-amqp \
--with-pic \
--with-librabbitmq-dir=yes \
CFLAGS=${CFLAGS} \
CHOST=\"${CHOST}\" "
eval $CFG_CMD

make -j `cat /proc/cpuinfo | grep processor | wc -l`

make install
${PHP_BIN_DIR}/php -n -dextension=${PHP_EXTENSIONS_DIR}/amqp.so --ri amqp

cd ..

This should build amqp.so that is linked agains our librabbitmq

ldd ${PHP_EXTENSIONS_DIR}/amqp.so |grep librabbitmq
#2:	librabbitmq.so.4 => /usr/local/php7/7.0.12/lib/rabbitmq-shared/lib/x86_64-linux-gnu/librabbitmq.so.4 (0x00007f910d263000)

Generate INI files

We are still missing the default .ini files, lets fix that, and make sure all extensions have linked correctly

ldd ${PHP_EXTENSIONS_DIR}/* | grep 'not found'

mkdir -p "${PHP_INI_CLI_SCAN_DIR}"

rm -f ${PHP_EXTENSIONS_DIR}/*a

find \
${PHP_EXTENSIONS_DIR}/*so \
'!' -name 'opcache.so' \
| sort \
| sed "s|$PHP_EXTENSIONS_DIR|extension=$PHP_EXTENSIONS_DIR|" \
> "${PHP_INI_CLI_EXTENSIONS}"

find \
${PHP_EXTENSIONS_DIR}/opcache.so \
| sed "s|$PHP_EXTENSIONS_DIR|zend_extension=$PHP_EXTENSIONS_DIR|" \
> "${PHP_INI_CLI_ZENDEXTENSIONS}"

\cp ${PHP_INI_FPM_FILE}.default ${PHP_INI_FPM_FILE}

mkdir -p "${PHP_INITD_DIR}"
cp sapi/fpm/init.d.php-fpm ${PHP_INITD_DIR}/php-fpm
chmod +x ${PHP_INITD_DIR}/php-fpm

Build deb package

To share our package, lets create a simplest deb package (27) (28)

#curl   uses system libcurl and openssl which are provided by `libcurl4-openssl-dev` (libcurl3)
#zlib   is provided by `zlib1g-dev`
#iconv  is provided by `libc-bin` (`/usr/bin/iconv`)
#libxml is provided by `libxml2-dev`

PHP_PACKAGE_DEB_NAME="mypackage-php-name"
PHP_PACKAGE_MAINTAINER="Sergei Shilko <contact@sshilko.com>"

cd ${PHP_PREFIX_DIR}/..
mkdir -p ${PHP_FULLVERSION}/DEBIAN
mkdir -p ${PHP_FULLVERSION}/${PHP_PREFIX_DIR}/
cp -rf ${PHP_PREFIX_DIR} ${PHP_FULLVERSION}/${PHP_PREFIX_DIR}/..

echo "Package: ${PHP_PACKAGE_DEB_NAME}
Version: ${PHP_VERSION}
Section: base
Priority: optional
Architecture: amd64
Depends: libcurl3, zlib1g (>= 1.0.9), libc-bin, libxml2 (>= 2.6.0)
Maintainer: ${PHP_PACKAGE_MAINTAINER}
Description: PHP" > ${PHP_FULLVERSION}/DEBIAN/control

dpkg-deb --build ${PHP_FULLVERSION}
mv ${PHP_FULLVERSION}.deb ${PHP_PACKAGE_DEB_NAME}-`date +"%F"`.deb
rm -rf ${PHP_FULLVERSION}

Benchmarking

Compare 7.0.10-2+deb.sury.org~trusty+1 vs our compiled 7.0.12 under ±5K RPM with NewRelic

Linux 3.13.0-74-generic x86_64
New Relic agent 2.2.0.125
Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz 4 cores 7.3 GB RAM

Response Time

CPU Time

Networking

#/usr/bin/php-config7.0
#7.0.10-2+deb.sury.org~trusty+1

--configure-options [--includedir=/usr/include --mandir=/usr/share/man --infodir=/usr/share/info --libdir=/usr/lib/x86_64-linux-gnu --libexecdir=/usr/lib/x86_64-linux-gnu --disable-maintainer-mode --disable-dependency-tracking --prefix=/usr --enable-cli --disable-cgi --disable-phpdbg --with-config-file-path=/etc/php/7.0/cli --with-config-file-scan-dir=/etc/php/7.0/cli/conf.d --build=x86_64-linux-gnu --host=x86_64-linux-gnu --config-cache --cache-file=/build/php7.0-DVHBcL/php7.0-7.0.10/config.cache --libdir=${prefix}/lib/php --libexecdir=${prefix}/lib/php --datadir=${prefix}/share/php/7.0 --program-suffix=7.0 --sysconfdir=/etc --localstatedir=/var --mandir=/usr/share/man --disable-all --disable-debug --disable-rpath --disable-static --with-pic --with-layout=GNU --without-pear --enable-filter --with-openssl=yes --with-pcre-regex=/usr --enable-hash --with-mhash=/usr --enable-libxml --enable-session --with-system-tzdata --with-zlib=/usr --with-zlib-dir=/usr --enable-dtrace --enable-pcntl --with-libedit=shared,/usr build_alias=x86_64-linux-gnu host_alias=x86_64-linux-gnu CFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -O2 -Wall -pedantic -fsigned-char -fno-strict-aliasing -g]

Compiling PHP 7.1.11 on Debian 9 x86_64

To be able to compile for debian 9 stretch minor changes need to be made

Update locale on clean image

apt-get install --no-install-recommends --no-upgrade -y locales libc-l10n
echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
locale-gen

Dependency changes:

libxslt-dev replaced with libxslt1-dev
libmysqlclient-dev replaced with default-libmysqlclient-dev
libpng12-dev replaced with libpng-dev
libedit-dev depencency added
libt1-dev removed

Build tools required, need to be installed manually

wget
tar
unzip
less

GCC 6 is the default one in debian 9, no need to install it

Literature

Written on October 14, 2016
https://moar.sshilko.com/2016/10/14/Compiling-Php