Home

Amazon Aurora Serverless MySQL – AWS RDS – LAMP

Leave a comment

Now a days big discussion going on serverless architecture, I have tested my database serverless scalable architecture for LAMP applications.

If you are looking to build serverless architecture for LAMP. Here is the guide how to design serverless architecture for any LAMP application.

So here is how to design / use scalable serverless MySQL database architecture.

I have divided my stack architecture in two part:

1.) Application Servers

2.) Database serversaws-aurora-serverless-mysql

 

I have used AWS EC2 for application server and AWS Aurora Serverless MySQL for database server.

 

So AWS EC2 instance stack look like below:

–Apache
–PHP
–Mysql client
–phpMyAdmin 
As I use Amazon Aurora Serverless MySQL, so don’t bother about database stack and scaling as Amazon Aurora Serverless MySQL automatically scales up & down and pause if not used.
You just have to configure min and max ACUs only else everything handles by amazon, means depending upon workload and number of concurrent connection you have to decide the min and max ACUs only.
I have used this tutorial for my application architecture.

 

I didn’t use Client Environment with Cloud9, instead I use EC2 micro instance with MySQL client only enough to connect aws aurora serverless.

 

Note : Mysql client only not mysql server

 

But I also used phpMyAdmin to verify it working properly for LAMP application.

 

So need to install Apache, PHP and phpMyAdmin and configured phpMyAdmin to connect database server remotely.

 

Here is how to install and configure Apache, PHP

 

https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-ubuntu-18-04

https://websiteforstudents.com/installing-apache2-mariadb-on-ubuntu-16-04-17-10-18-04-with-php-7-2-support-lamp/

And here is how to install and configured phpMyAdmin to connect database server remotely.

https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-phpmyadmin-on-ubuntu-18-04

https://websiteforstudents.com/manage-remote-mysql-databases-via-phpmyadmin-on-ubuntu-17-04-17-10/

https://stackoverflow.com/questions/16801573/how-to-access-remote-server-with-local-phpmyadmin-client

Advantages of Aurora Serverless:
Aurora Serverless provides the following advantages:

Simpler
Aurora Serverless removes much of the complexity of managing DB instances and capacity.

Scalable
Aurora Serverless seamlessly scales compute and memory capacity as needed, with no disruption to client connections.

Cost-effective
When you use Aurora Serverless, you pay for only the database resources that you consume, on a per-second basis.

Highly available storage
Aurora Serverless uses the same fault-tolerant, distributed storage system with six-way replication as Aurora to protect against data loss.

How Aurora Serverless works?
aurora-serverless-arch
Full detail can be found here : how it works

Aurora Capacity Units (ACUs) 

Database Capacity

In Aurora Serverless, database capacity is measured in Aurora Capacity Units (ACUs). 1 ACU has approximately 2 GB of memory with corresponding CPU and networking, similar to what is used in Aurora Standard instances. You pay a flat rate per second of ACU usage, with a minimum of 5 minutes of usage each time the database is activated. 

1 Aurora Capacity Unit (ACU) $0.06 per ACU Hour
As your database capacity is automatically scaled up or down, ACUs are added or removed. Aurora Serverless can scale from a minimum of 2 ACUs to a maximum of 256 ACUs. You can specify the minimum and maximum ACUs your database can consume.

Database Storage & I/O

Storage consumed by your Amazon Aurora database is billed in per GB-month increments and I/Os consumed are billed in per million request increments. You pay only for the storage and I/Os your Amazon Aurora database consumes and do not need to provision in advance. Amazon Aurora storage and I/O prices are the same for Standard and Serverless configurations.

Storage Rate $0.10 per GB-month
I/O Rate $0.20 per 1 million requests
Full details can be found here:

Limitations of Aurora Serverless

The following limitations apply to Aurora Serverless:

Note

You can access an Aurora Serverless DB cluster from AWS Lambda. For more information about working with AWS Lambda, see Configuring a Lambda Function to Access Resources in an Amazon VPC in the AWS Lambda Developer Guide.

 

More information can be found here:

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html

https://aws.amazon.com/rds/aurora/serverless/

https://aws.amazon.com/rds/aurora/

https://aws.amazon.com/blogs/aws/aurora-serverless-ga/

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.create.html

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.CreateInstance.html

https://www.jeremydaly.com/aurora-serverless-the-good-the-bad-and-the-scalable/

View story at Medium.com

 

Advertisements

Web Admin Backend Development with UI : Laravel Admin Panel

Leave a comment

Any web app required a very flexible and extensible way of building the custom Laravel 5 applications. Like : administrative tasks – Users, Roles and Permissions management etc. Mostly we require a backend interface to make the application.

So here are the some good Open Source Laravel admin system to build a good backend application with good admin theme/UI.

1.) Laravel 5 Boilerplate / Starter kit with Gentelella Admin Theme
Laravel Boilerplate provides a very flexible and extensible way of building your custom Laravel 5 applications.68747470733a2f2f63646e2e636f6c6f726c69622e636f6d2f77702f77702d636f6e74656e742f75706c6f6164732f73697465732f322f67656e74656c656c6c612d61646d696e2d74656d706c6174652d707265766965772e6a7067
Full detail feature available here
https://github.com/Labs64/laravel-boilerplate
https://colorlib.com/polygon/gentelella/index.html

2.) An other Laravel 5.7 Boilerplate with CoreUI Admin Theme
Laravel Boilerplate provides us with a massive head start on any size web application. It comes with a full featured access control system out of the box with an easy to learn API and is built on a Bootstrap foundation with a front and backend architecture.default-layout
Full detail feature available here
https://github.com/rappasoft/laravel-5-boilerplate
http://laravel-boilerplate.com/5.7/start.html
https://coreui.io/#live-preview

3.) Voyager : Laravel Admin Package
a Laravel Admin Package that includes BREAD(CRUD) operations, a media manager, menu builder, and much more. It will take care of all administrative tasks, this way we can focus on what we do best, which is building the next kick-ass app!

features-media-managerfeatures-menu-builderfeatures-bread-builderfeatures-settings
Full detail feature available here
https://laravelvoyager.com/
https://github.com/the-control-group/voyager
https://voyager.readme.io/docs/installation

4.) LaraAdmin: a Laravel CRM for quick-start Admin based applications
LaraAdmin is a Open source Laravel Admin Panel / CMS which can be used as Admin Backend, Data Management Tool or CRM boilerplate for Laravel with features like Advanced CRUD Generation, Module Manager, Backups and many more.
LaraAdmin is a Open source CRM for quick-start Admin based applications with features like Advanced CRUD Generation, Schema Manager and Workflows.
LaraAdmin is a remarkable package offering us a complete set of utilities we will ever need as a CRM / Admin Panel. Whether creating a CMS, Backend Panel or CRM – LaraAdmin’s unique and flexible utilities has covered
LaraAdmin comes with quite a few predefined modules. They help us to get to know the system, but, on the other hand, if I had to create my project from scratch, it would take me some time to get rid of those LaraAdmin defaults.
Snapshots:


Full detail feature available here
http://laraadmin.com/
https://github.com/dwijitsolutions/laraadmin

5.) Quick adminpanel builder package for Laravel 5 : admin panel generator QuickAdminPanel: Quick adminpanel builder package for Laravel 5quickadmin-1demo-appointments-01

Full detail feature available here
https://quickadminpanel.com/
https://github.com/LaravelDaily/quickadmin
http://docs.quickadminpanel.com/Download_and_Install_Your_Panel/General_Instructions.html
Vue.js+Laravel version of generator: Vue.QuickAdminPanel.com
Here a good article we can read for Laravel Admin
13 Laravel Admin Panel Generators
https://laravel-news.com/13-laravel-admin-panel-generators

Technologies Every Web Architect/Developer/Engineer Should Know

Leave a comment

Client side technologies

Front-end technologies

–Browsers

–HTML5

–CSS3

–CSS3/LESS/SASS

Programming Languages – JS

–JavaScript/TypeScript/CoffeeScript

Front-end Libraries – JS

–jQuery/jQueryUI/UnderscoreJS/RequireJS

–C3.js/D3.js

Front-end frameworks – JS

–AngularJS/ReactJS/EmberJS/BackboneJS/VueJS

–Phonegap/Cordova/Ionic

CSS Frameworks

–Bootstrap

–Foundation

Server side technologies

Back-end technologies

–Apache web server

–Nginx

–NodeJS

–Lighthttpd

Databases

–PostgreSQL/MySQL/MariaDB/SQLite

NoSQL

–MongoDB/CouchDB/Redis/Memcached

Programming Languages

–Python/PHP/JAVA/Ruby/Go/Erlang/Elixir

Back-end frameworks

–Django/Laravel/CodeIgniter/Symfony/Zend

Cloud

–AWS/Azure/GCP

————————————————————–

IDEs:

–Netbeans

–eclipse

–Notepad++

Tools:

–composer

–bower

–gulp

–django – orm/querysets

–django – admin/manage

–laravel elixir/mix

–laravel artisan

–laravel – eloquent

–laravel – blade

–SQLAlchemy

–Celery

–ORM

–templating

 

HTML5/HTML

HTML 5 is a markup language used for structuring and presenting content on the World Wide Web. It is the fifth and current major version of the HTML standard, and subsumes XHTML

CSS3/CSS

CSS3 is the latest evolution of the Cascading Style Sheets language and aims at extending CSS2.1. It brings a lot of long-awaited novelties, like rounded corners, shadows, gradients, transitions or animations, as well as new layouts like multi-columns, flexible box or grid layouts.

https://developer.mozilla.org/en-US/docs/Web/CSS/CSS3

https://developer.mozilla.org/en-US/docs/Web/CSS

LESS– a CSS pre-compiler to make working with CSS easier and add functionality

Less (which stands for Leaner Style Sheets) is a backwards-compatible language extension for CSS. This is the official documentation for Less, the language and Less.js, the JavaScript tool that converts your Less styles to CSS styles.
SASS– a CSS pre-compiler to make working with CSS easier and add functionality [CSS with superpowers]

Sass is a style sheet language initially designed by Hampton Catlin and developed by Natalie Weizenbaum. After its initial versions, Weizenbaum and Chris Eppstein have continued to extend Sass with SassScript, a simple scripting language used in Sass files.

Programming Languages

Javascript– used by all web browsers, and lots of other frameworks.

JavaScript, often abbreviated as JS, is a high-level, interpreted programming language. It is a language which is also characterized as dynamic, weakly typed, prototype-based and multi-paradigm. Alongside HTML and CSS, JavaScript is one of the three core technologies of the World Wide Web.

Coffeescript– is a kind of “dialect” of javascript. It is viewed as simpler and easier on your eyes as a developer but it complies (converts) back into javascript

Typescript is an open-source programming language developed and maintained by Microsoft. It is a strict syntactical superset of JavaScript, and adds optional static typing to the language. TypeScript is designed for development of large applications and transcompiles to JavaScript.

PHPPHP: Hypertext Preprocessor is a server-side scripting language designed for Web development, but also used as a general-purpose programming language.

PythonPython is an interpreted high-level programming language for general-purpose programming and used by the Django framework and used in a lot of mathematical calculations.

Erlang– is a general-purpose, concurrent, functional programming language, as well as a garbage-collected runtime system.

JS Libraries:

jQuery

http://jquery.com/

jQueryUI

https://jqueryui.com/

Underscore

https://underscorejs.org/

Require

https://requirejs.org/

JS Frameworks:

Bootstrap– a UI (user interface) framework for building with HTML/CSS/Javascript

Angular.js– a front-end javascript framework.

React.js– A JavaScript library for building user interfaces

React makes it painless to create interactive UIs. Design simple views for each state in your application, and React will efficiently update and render just the right components when your data changes.

Ember.js– a front-end javascript framework.

Backbone.js– a front-end javascript framework.

Phonegap / Cordova– a mobile framework that exposes native api’s of iOS and Android for use when writing javascript

Ionic– a mobile framework similar to phonegap/cordova

Foundation– a UI framework for building with HTML/CSS/Javascript

Meteor– a full-stack (front and back end) javascript framework

https://docs.meteor.com/

Vue.js

Vue.js is a JavaScript web application framework for building rich apps that run in web browsers.

Backend PHP/Python Frameworks:

Laravel is a free, open-source PHP web framework and intended for the development of web applications following the model–view–controller (MVC) architectural pattern and based on Symfony.

https://laravel.com/

Symfony is a set of reusable PHP components and a PHP framework to build web applications, APIs, microservices and web services.

https://symfony.com/

CodeIgniter is a powerful PHP framework with a very small footprint, built for developers who need a simple and elegant toolkit to create full-featured web applications.

https://www.codeigniter.com/

Django– a full-stack framework built using python

Django makes it easier to build better Web apps more quickly and with less code. Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of Web development, so you can focus on writing your app without needing to reinvent the wheel. It’s free and open source.

Node.js a server-side javascript framework

https://nodejs.org/en/

Databases:

MongoDBis a free and open-source cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with schemata.

Redis– is the most popular key-value store. It is lighting fast for retrieving data but doesn’t allow for much depth in the data storage.

Redis, RE-dis is an open-source in-memory data structure project implementing a distributed, in-memory key-value database with optional durability. Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, hyperloglogs, bitmaps and spatial indexes.

Memcachedis a general-purpose distributed memory caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source must be read.

PostgreSQL– is a popular open-sourced SQL database.

MySQL– is another popular open-sourced SQL database. MySQL is used in WordPress websites.

MariaDB-is a community-developed fork of the MySQL relational database management system.

Tools:

Bower: A package manager for the web

https://bower.io/

Composer : Dependency Manager for PHP

Composer is an application-level package manager for the PHP programming language that provides a standard format for managing dependencies of PHP software and required libraries.

https://getcomposer.org/

gulp is an open-source JavaScript toolkit by Fractal Innovations and the open source community at GitHub, used as a streaming build system in front-end web development.

Gulp is a toolkit for automating painful or time-consuming tasks in your development workflow, so you can stop messing around and build something.

https://gulpjs.com/

Data formats

JSON– is quickly becoming the most popular data format

XML– was the main data format early in the web days and predominantly used by Microsoft systems

CSV– is data formatted by commas. Excel data is typically formatted this way.

Others:

REST– is a protocol mainly used for API’s. It has standard methods like GET, POST, and PUT that let information be exchanged between applications.

Object-Relational Mapper (ORM)

Laravel ORM – Eloquent

https://laravel.com/docs/5.7/eloquent

Django Object-Relational Mapper (ORM)

https://www.fullstackpython.com/django-orm.html

SQLAlchemy

SQLAlchemy (source code) is a well-regarded database toolkit and object-relational mapper (ORM) implementation written in Python. SQLAlchemy provides a generalized interface for creating and executing database-agnostic code without needing to write SQL statements.

Task queues : Distributed Task Queue

Task Worker

Celery

Celery is a task queue implementation for Python web applications used to asynchronously execute work outside the HTTP request-response cycle.

Celery: Distributed Task Queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Celery is used in production systems to process millions of tasks a day.

Redis Queue (RQ)

Redis Queue (RQ) is a Python task queue implementation that uses Redis to keep track of tasks in the queue that need to be executed.

Task Scheduler

Celery Beat

http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html

celery beat is a scheduler; It kicks off tasks at regular intervals, that are then executed by available worker nodes in the cluster. By default the entries are taken from the beat_schedule setting, but custom stores can also be used, like storing the entries in a SQL database.

Task Monitoring

Flower – Celery monitoring tool

https://flower.readthedocs.io/en/latest/

Real-time monitoring using Celery Events. Task progress and history; Ability to show task details (arguments, start time, runtime, and more); Graphs and statistics.

Celery Flower

The Celery Flower is a tool for monitoring your celery tasks and workers. It’s web based and allows you to see task progress, details, worker status.

Templating Engine:

Blade Templating. Blade is a simple, yet powerful templating engine provided with Laravel. Unlike controller layouts, Blade is driven by template inheritance and sections. All Blade templates should use the .blade.php extension.

Blade is the simple, yet powerful templating engine provided with Laravel. Unlike other popular PHP templating engines, Blade does not restrict you from using plain PHP code in your views. In fact, all Blade views are compiled into plain PHP code and cached until they are modified, meaning Blade adds essentially zero overhead to your application. 

https://laravel.com/docs/5.7/blade

https://laravel.com/docs/5.0/templates

Bootstrap Responsive Admin Templates – Material Admin

https://wrapbootstrap.com/theme/material-admin-responsive-admin-theme-WB011H985

Open Source Server/Application Software Stacks

Leave a comment

Open Source Server/Application Software Stacks

LAMP (software bundle)

LAMP/MAMP/WAMP – Linux/Mac/Windows, Apache, MySQL/MariaDB, PHP/Python/Perl
LLMP – Linux, Lighttpd, MySQL/MariaDB, PHP/Python/Perl
LAPP – Linux, Apache, PostgreSQL, PHP/Python/Perl
LEMP/LNMP – Linux, Nginx, MySQL/MariaDB, PHP/Python/Perl

MEAN (software bundle)

MEAN/MEEN/MERN/MEVN – MongoDB, Express.js, Angular.js/Ember.js/React.JS/Vue.js, Node.js
MongoDB, a NoSQL database
Express.js, a web application framework that runs on Node.js
Angular.js or Angular, JavaScript MVC frameworks that run in browser JavaScript engines
Node.js, an execution environment for event-driven server-side and networking applications

LYME (software bundle)

LYME/LYCE – Linux, Yaws, Mnesia/CouchDB, Erlang
Linux, the operating system;
Yaws, the web server;
Mnesia or CouchDB, the database;
Erlang, the functional programming language.

 

LAMP

Linux (operating system)
Apache (web server)
MySQL or MariaDB (database management systems)
Perl, PHP, or Python (scripting languages)

LAPP

Linux (operating system)
Apache (web server)
PostgreSQL (database management systems)
Perl, PHP, or Python (scripting languages)

LEMP/LNMP

Linux (operating system)
Nginx (web server)
MySQL or MariaDB (database management systems)
Perl, PHP, or Python (scripting languages)

LLMP

Linux (operating system)
Lighttpd (web server)
MySQL or MariaDB (database management systems)
Perl, PHP, or Python (scripting languages)

LYME and LYCE

Linux (operating system)
Yaws (web server, written in Erlang)
Mnesia or CouchDB (database, written in Erlang)
Erlang (functional programming language)

MAMP

Mac OS X (operating system)
Apache (web server)
MySQL or MariaDB (database)
PHP, Perl, or Python (programming languages)

MEVN

MongoDB (database)
Express.js (app controller layer)
Vue.js (web app presentation)
Node.js (web server)

WAMP

Windows (operating system)
Apache (web server)
MySQL or MariaDB (database)
PHP, Perl, or Python (programming language)

WIMP

Windows (operating system)
Internet Information Services (web server)
MySQL or MariaDB (database)
PHP, Perl, or Python (programming language)

XAMPP

cross-platform (operating system)
Apache (web server)
MariaDB or MySQL (database)
PHP (programming language)
Perl (programming language)

Best JavaScript Frameworks (Libraries)

1 Comment

Best JavaScript Frameworks/Libraries:

1.) JQuery

2.) YUI

3.) Dojo

4.) ExtJS

5.) Prototype

6.) MooTools

7.) script.aculo.us

8.) AngularJS

9.) Backbone.js

10.)Ember.js

References:

http://www.w3schools.com/js/js_libraries.asp

http://en.wikipedia.org/wiki/Comparison_of_JavaScript_frameworks

http://readwrite.com/2014/02/06/angular-backbone-ember-best-javascript-framework-for-you

http://www.infoq.com/research/top-javascript-mvc-frameworks

http://jonathanmh.com/best-javascript-mvc-frameworks-2013-2014/

http://www.infoworld.com/d/application-development/the-10-hottest-javascript-framework-projects-228335

http://designzum.com/2014/02/19/top-best-javascript-framework-2014/

http://code.tutsplus.com/articles/20-javascript-frameworks-worth-checking-out–net-22020

How to use Couchdb NoSQL database on CodeIgniter

Leave a comment

Here I will explain how to use CouchDB NoSQL database on CodeIgniter using PHP On Couch open source library (Data Access Library to access a CouchDB server with PHP).

PHP On Couch tries to provide an easy way to work with your CouchDB documents with PHP.

Here I assume you know PHP and CodeIgniter. If don’t go through http://ellislab.com/codeigniter Its running on your system.

Here is How to use Couchdb NoSQL database on CodeIgniter?

First Download PHP On Couch from here:

https://github.com/dready92/PHP-on-Couch

Setup instructions:

https://github.com/dready92/PHP-on-Couch/tree/master/frameworks/ci

1.) copy the PHP on Couch classes

copy all from /PHP-on-Couch-master/lib/

to /CodeIgniter_2.1.4/application/libraries/

2.) copy the PHP on Couch CI classes

copy from /PHP-on-Couch-master/frameworks/ci/libraries/

to /CodeIgniter_2.1.4/application/libraries/

3.) copy the config/couchdb.php file inside the config folder of your application/libraries/

copy from /PHP-on-Couch-master/frameworks/ci/config

to /CodeIgniter_2.1.4/application/config/

4.) edit the config/couchdb.php file to set the two configuration options couch_dsn and couch_database :

$config[‘couch_dsn’] = “http://localhost:5984/“;

$config[‘couch_database’] = “my_wonderful_db”;

5.) if you want the couchdb object to be autoloaded, edit the config/autoload.php file of your CodeIgniter application and add “couchdb” to the list of libraries to autoload

$autoload[‘libraries’] = array(‘couchdb’);

6.) Thats it!

References:

https://github.com/dready92/PHP-on-Couch/blob/master/doc/couch_client-database.md

https://github.com/dready92/PHP-on-Couch/blob/master/doc/couch_client-document.md

https://github.com/dready92/PHP-on-Couch/tree/master/doc

http://wiki.apache.org/couchdb/Getting_started_with_PHP

http://en.wikipedia.org/wiki/CouchDB

http://couchdb.apache.org/

http://planet.couchdb.org/

http://guide.couchdb.org/editions/1/en/index.html

http://docs.couchdb.org/en/latest/

http://wiki.apache.org/couchdb/

How to Install XHProf function-level hierarchical profiler for PHP on CodeIgniter

Leave a comment

First what is XHProf?

From: https://github.com/facebook/xhprof
XHProf is a function-level hierarchical profiler for PHP and has a simple HTML based user interface. The raw data collection component is implemented in C (as a PHP extension). The reporting/UI layer is all in PHP. It is capable of reporting function-level call counts and inclusive and exclusive wall time, CPU time and memory usage.

From: http://pecl.php.net/package/xhprof
XHProf: A Hierarchical Profiler for PHP
XHProf is a function-level hierarchical profiler for PHP and has a simple HTML based navigational interface. The raw data collection component is implemented in C (as a PHP extension). The reporting/UI layer is all in PHP. It is capable of reporting function-level inclusive and exclusive wall times, memory usage, CPU times and number of calls for each function. Additionally, it supports ability to compare two runs (hierarchical DIFF reports), or aggregate results from multiple runs.

There are a number of ways to profile and monitor the performance of PHP code in a development environment, but once it’s been let loose on a production server its extremely difficult to know what’s going on when your app is getting accessed by your users.
In March 2009, Facebook released XHProf, a function level profiler that allows you to identify bottlenecks in your application. And it runs on a production server with little overhead.
XHProf will provide you with a hierarchical profile of the functions your application calls, allowing you to follow the chain of events that lead up to resource intensive code running. You’ll be able to see exactly how much wall time (the actual time elapsed), CPU time and memory your application uses.
This tutorial will help you get XHProf installed on your server, and integrated with your CodeIgniter application via hooks so that you can start profiling your application, in a production environment, straight away. Everything you see here was written for Ubuntu 12.04:

Installation of XHProf can be done via PECL – as per php manual, it’s very hectic get the PECL version installed on Ubuntu. For me it’s easier to install from source.

So here I will explain how to install from source. To install from source we will require “phpize” pecl extension. So first we will install phpize.

In this topic I am assuming a PHP web server running with (Apacpe and PHP).

Step 1.) How to install phpize on ubuntu?

On Debian/Ubuntu, it’s in the php5-dev package, so we will install php dev package.

$ sudo apt-get install php5-dev

It will be available here: /usr/bin/phpize instead of only phpize.

The php5-dev package contains various files absolutely essential to compiling PHP. Since it’s a shell script, you can just copy it from the package, but it won’t work without the dependencies for building a PHP extensions (like header files).

Explanation

In the C world, compiling dynamically linked libraries (DLL/SO) is done by using header files that contain the description on how the libraries are to be interacted with (function calls, structures, etc). These are used to compile an extension without the need for the PHP source code itself. In other words, you don’t need the PHP source but you actually need the header files which are contained in php5-dev.

You can check with phpize cmd

Step 2.) How to install XHProf on ubuntu

To install from source first download source from github (https://github.com/facebook/xhprof/tarball/master)

$ wget https://github.com/facebook/xhprof/tarball/master -O xhprof.tar.gz

if you get error :

–2013-10-28 18:30:29– https://github.com/facebook/xhprof/tarball/master
Resolving github.com (github.com)… 192.30.252.130
Connecting to github.com (github.com)|192.30.252.130|:443… connected.
ERROR: cannot verify github.com’s certificate, issued by `/C=91/ST=UP/L=Noida/O=SRI/OU=SRI/CN=SRI/emailAddress=rakesh.snr@samsung.com’:
Unable to locally verify the issuer’s authority.
To connect to github.com insecurely, use `–no-check-certificate’.

then use this

$ wget --no-check-certificate https://github.com/facebook/xhprof/tarball/master -O xhprof.tar.gz

Extract the tarball and switch into the extracted folder – this will change depending on the latest revision available from GitHub. You can directly download and extract it.

$ tar -xvf xhprof.tar.gz
$ cd facebook-xhprof-ec8b884/extension

Then install as normal.

$ phpize
$ ./configure
$ make
$ sudo make install

Now copy xhprof source to web dir (doc root folder) for viewing the logs and make dog dir inside it.

$ sudo cp -r facebook-xhprof-ec8b884/  /var/www/xhprof/
$ sudo mkdir /var/www/xhprof/log_data
$ sudo chmod -R 777 /var/www/xhprof/log_data/

Note: path of xhproff log dir must be php execute user accessible. Means it must be in /var/www/ dir or accessible to www-dada user as this www-data user execute php script and xhprof log are created by this www-data user.

Finally, we need to tell PHP to load the extension. Create a config file for it – I do this for cleanliness but you can always drop it at the end of your main php.ini.

$ sudo gedit /etc/php5/conf.d/xhprof.ini

And paste in the following:

extension=xhprof.so
xhprof.output_dir="/var/www/xhprof/log_data"

This tells XHProf to use the directory at /var/www/xhprof/log_data to save its run data.
Note: path of xhproff log dir must be php execute user accessible. Means it must be in /var/www/ dir or accessible to www-dada user as this www-data user execute php script and xhprof log are created by this www-data user.
You can check whether XHProf is installed correctly by entering php -m at the command line and checking that the module is available. Remember to restart Apache so that it gets picked up in your PHP web applications as well.

If you want to render the callgraph images you’ll also need the graphviz package. This can be obtained from apt.

$ sudo apt-get install graphviz

Now xhprof ready
With front end – http://localhost/xhprof/xhprof_html/

Home-All-1-XHProf_ Hierarchical Profiler Report - 2013-10-29_17.14.21

Step 3.) Finally how to Integration XHProf with CodeIgniter

First setup codeigniter:
Download codeigniter and extract it in home folder
http://ellislab.com/codeigniter
Copy code to web root folder to make accessible to apache with localhost

$ sudo cp -R CodeIgniter_2.1.4 /var/www/

Now open from localhost CI URL
http://localhost/CodeIgniter_2.1.4/

CodeIgniter

If this works then CI setup ready

XHProf can be used on an ad-hoc basis to evaluate small pieces of code, but it’s at its most useful when you let it profile the full page. First of all, you’ll need to move the XHProf code into your web applications root so that it has access to the appropriate classes.
This already explained in 2nd setup, but again I will explain it.

Copy xhprof source to web dir (doc root folder) for viewing the logs and make log dir inside it or inside doc root to make www-data user accessible and also set writable permission to www-data user. This has been done in 2nd step so it must not be done again.

$ sudo cp -r facebook-xhprof-ec8b884/  /var/www/xhprof/
$ sudo mkdir /var/www/xhprof/log_data
$ sudo chmod -R 777 /var/www/xhprof/log_data/

Note: path of xhproff log dir must be php execute user accessible. Means it must be in /var/www/ dir or accessible to www-dada user as this www-data user execute php script and xhprof log are created by this www-data user.

http://localhost/xhprof/xhprof_html/

CodeIgniter has an excellent mechanism for injecting custom code into the execution of a page called ‘Hooks’. This is what we’ll use to integrate XHProf with your application.
Enable hooks in your application/config/config.php file.

$ sudo gedit /var/www/CodeIgniter_2.1.4/application/config/config.php

application/config/config.php file :
//set enable_hooks to true

$config['enable_hooks'] = true;

// and

/*
 XhProf config
*/
$config['xhprof_lib_dir'] = '/var/www/xhprof/log_data/';
$config['xhprof_ui_url'] = 'http://localhost/xhprof/xhprof_html/index.php';
$config['xhprof_flag'] = TRUE; // TRUE / FALSE to enable and disable

Then specify your hooks in application/config/hooks.php

$ sudo gedit /var/www/CodeIgniter_2.1.4/application/config/hooks.php

application/config/hooks.php file:

<?php if ( ! defined('BASEPATH')) exit('No direct script access allowed');
/*
| -------------------------------------------------------------------------
| Hooks
| -------------------------------------------------------------------------
| This file lets you define "hooks" to extend CI without hacking the core
| files. Please see the user guide for info:
|
| http://codeigniter.com/user_guide/general/hooks.html
|
*/

$hook['pre_controller'] = array(
 'class' => 'XHProf',
 'function' => 'XHProf_Start',
 'filename' => 'xhprof.php',
 'filepath' => 'hooks',
 'params' => array()
);

$hook['post_controller'] = array(
 'class' => 'XHProf',
 'function' => 'XHProf_End',
 'filename' => 'xhprof.php',
 'filepath' => 'hooks',
 'params' => array()
);

$hook['post_system'] = array(
 'class' => 'XHProf',
 'function' => 'XHProf_list_runs',
 'filename' => 'xhprof.php',
 'filepath' => 'hooks',
 'params' => array()
);

/* End of file hooks.php */
/* Location: ./application/config/hooks.php */

Then create the hook that will load XHProf into your application at application/hooks/xhprof.php. This class will provide the necessary minimum to get XHProf collecting data from your application.

$ sudo gedit /var/www/CodeIgniter_2.1.4/application/hooks/xhprof.php

application/hooks/xhprof.php file:

<?php if ( ! defined('BASEPATH')) exit('No direct script access allowed');

class XHProf {

 private $XHProfPath = '/var/www/xhprof/';
 private $applicationName = 'my_xhprof_CI';
 private $sampleSize = 1;
 private static $enabled = false;
 private $suffix = 'xhprof';
 private static $xhprof_flag = false;

var $CI;

 public function XHProf_Start() {
 self::$xhprof_flag = config_item('xhprof_flag');
 if (self::$xhprof_flag && mt_rand(1, $this->sampleSize) == 1) {
 include_once $this->XHProfPath . 'xhprof_lib/utils/xhprof_lib.php';
 include_once $this->XHProfPath . 'xhprof_lib/utils/xhprof_runs.php';
 xhprof_enable(XHPROF_FLAGS_NO_BUILTINS);
 self::$enabled = true;
 }
 }

 public function XHProf_End() {
 if (self::$xhprof_flag && self::$enabled) {
 $XHProfData = xhprof_disable();

$XHProfRuns = new XHProfRuns_Default();
 $this->CI =& get_instance();

$name = $this->applicationName;
 $className = $this->CI->router->class;
 if(!empty($className))
 $name .= '_' . $className;
 $methodName = $this->CI->router->method;
 if(!empty($methodName))
 $name .= '_' . $methodName;

$XHProfRuns->save_run($XHProfData, $name);
 }
 }

function XHProf_list_runs() {
 if (self::$xhprof_flag && self::$enabled) {

$this->CI =& get_instance();

$name = $this->applicationName;
 $className = $this->CI->router->class;
 if(!empty($className))
 $name .= '_' . $className;
 $methodName = $this->CI->router->method;
 if(!empty($methodName))
 $name .= '_' . $methodName;

 $xhprof_lib_dir = config_item('xhprof_lib_dir');
 $xhprof_ui_url = $this->CI->config->item('xhprof_ui_url');

if (is_dir($xhprof_lib_dir)) {

 echo "<hr/>Current runs:\n<ul>\n";
 $files = glob("{$xhprof_lib_dir}/*.$name.{$this->suffix}");
 usort($files, create_function('$a,$b', 'return filemtime($b) - filemtime($a);'));
 $f = true;
 foreach ($files as $file) {

 list($run,$source) = explode('.', basename($file));
 echo '<li><a href="' . htmlentities($xhprof_ui_url)
 . '?run=' . htmlentities($run) . '&source='
 . htmlentities($source) . '" target="_blank">'
 . htmlentities(basename($file)) . "</a><small> "
 . date("Y-m-d H:i:s", filemtime($file)) . "</small></li>\n";
 if($f)
 {
 $f = false;
 echo "</ul>\n<hr/>Older runs:\n<ul>\n";
 }
 }
 echo "</ul>\n";
 }
 }

 }

}

?>

Now open from localhost CI URL
http://localhost/CodeIgniter_2.1.4/

Welcome to CodeIgniter - 2013-10-29_17.16.59

If you see this then it works

There’s a few thing to point out in this code sample.

• The $XHProfPath variable should point to the directory you installed XHProf in as explained in step 2 and also above in this step. In our example, we put it in the root of the web application, but you might be storing it somewhere central and symlinking it to multiple applications.
• The $applicationName variable lets you specify the name of the application using XHProf. This could be especially important in an environment where you’re running multiple applications on the same server.
• The $sampleSize variable lets you specify a factor for how often XHProf profiles a run. In a production environment where you’re receiving thousands of hits, it’s probably not worth storing every single run. You can increase this variable to get a random sample of runs. Changing it to 10, for example, will give you a report from one in every 10 visits to your application.

The xhprof_enable() function can accept any combination of 3 constants.

• XHPROF_FLAGS_NO_BUILTINS – skip all internal php functions. Adding this means you’ll only see time spent in functions that you have written (or are part of CodeIgniter)
• XHPROF_FLAGS_CPU – add additional CPU profiling information
• XHPROF_FLAGS_MEMORY – add additional memory profiling information
• Combine them using +. eg. xhprof_enable(XHPROF_FLAGS_NO_BUILTINS + XHPROF_FLAGS_MEMORY);
After running through your application a couple of times, point your browser at the XHProf application directory http://localhost/xhprof/xhprof_html/index.php – adjusting for the address of your development server – and you’ll see a list of your last application runs with the newest at the top. Select one of the runs to see its profiling information.

Home-All-XHProf_ Hierarchical Profiler Report - 2013-10-29_17.14.21

Evaluating the Results
Storing all this profiling information is all very well and good but knowing how to evaluate it is the key to optimizing your application.
The ‘Overall Summary’ is a good place to start – it’ll show you:
• Total inclusive wall time – how long (in microseconds) it took for you page to be generated
• Total memory use – the total memory used by this run of your application
• Total peak memory use – the maximum amount of memory that was used by your application
• Number of function calls – the number of functions that were called by your application

XHProf_ Hierarchical Profiler Report - 2013-10-29_17.17.24

These numbers should give you an overall base to start from and provide headline numbers to compare against when you start optimization.
Below the summary is the detailed run report the bottlenecks in your application should be fairly obvious – they’re the function calls at the top of the report.

The most resource intensive things you’re likely to come across will be those that make calls to external services or your database. By default, the results are sorted by ‘inclusive wall time’. This shows the functions that took the longest to run by the cumulative total of them and the functions they call. Ordering by ‘exclusive wall time’ allows you to see the specific functions that are using the most amount of resource.
Clicking a function allows you to drill down into more detail about that function and the functions it called. From there, it’ll be easier to see exactly what your long running function was doing and spot any issues.
When debugging your CodeIgniter application, one of the best places to start is the controller being called for your page – Welcome::index in the example. From there you can easily see the breakdown of where the controller spent most of its time.

Welcome-Index-XHProf_ Hierarchical Profiler Report - 2013-10-29_17.19.29

Callgraph
XHProf can generate a call graph view that provides a visual clue to the main path of your application. The main bottlenecks are highlighted in red to show where most your resources are being spent.

callgraph.php (PNG Image, 2945 × 1779 pixels) - 2013-10-29_17.18.51

Comparing Runs
XHProf provides a mechanism for comparing multiple runs – this will allow you to make a number of optimizations and then see how they’ve affected your application.
http://localhost/xhprof/xhprof_html/index.php?run1=run_id_1>&run2=run_id_2&source=applicationName
Note: run_id_1 and run_id_2 should be the ids from some of your previous runs and applicationName should be the name you specified when setting up the hook earlier.
This will provide the same information you see in a regular report but provide percentage statistics on any changes since the first run, allowing you to easily see if your optimizations are moving your performance in the right direction.

http://localhost/xhprof/xhprof_html/index.php?run1=526fa02b94f31&run2=526fa0255926b&source=my_xhprof_CI_welcome_index

XHProf_ Hierarchical Profiler Report - 2013-10-29_17.30.43

Aggregating Runs
Sometimes a single run might not be enough to evaluate exactly what’s going on. Pages my work slightly differently depending on the query string or user input or there might be differences in caching mechanisms.
Aggregating a number of runs allows you to combine a number of runs and receive an average of the resources consumed.
http://localhost/xhprof/xhprof_html/index.php?run=run_id,run_id,run_id&source=applicationName
Note: run_id should be the ids from some of your previous runs and applicationName should be the name you specified when setting up the hook earlier.

http://localhost/xhprof/xhprof_html/index.php?run=526fa02b94f31,526fa0255926b&source=my_xhprof_CI_welcome_index

XHProf_ Hierarchical Profiler Report - 2013-10-29_17.41.01

Summary
You should now have XHProf up and running in your CodeIgniter application and be able to see exactly where your code is spending most of it’s time. Optimisation can be a difficult task. Sometimes it’s not as simple as reimplementing a function, or adding a layer of caching. Thinking about why you’re doing something and what the exact effect that is having on your application is just as important.
So let me know if you have any questions in the comments below. Thank you so much for reading!

References:

http://net.tutsplus.com/tutorials/php/advanced-codeigniter-profiling-with-xhprof/
https://github.com/facebook/xhprof
http://www.catswhocode.com/blog/how-to-create-a-simple-and-efficient-php-cache
http://css-tricks.com/snippets/php/intelligent-php-cache-control/
http://www.phptoys.com/tutorial/speed-up-with-caching.html
http://www.addedbytes.com/articles/for-beginners/output-caching-for-beginners/
Xcache: http://xcache.lighttpd.net/
APC: http://php.net/manual/en/book.apc.php
Varnish: https://www.varnish-cache.org/
http://www.sitepoint.com/getting-started-with-varnish/
Even More: http://en.wikipedia.org/wiki/List_of_PHP_accelerators

Older Entries

%d bloggers like this: