:::: MENU ::::

Aadhaar Notes – Part I: Some Quick Fixes

The Aadhaar (TARGETED DELIVERY OF FINANCIAL AND OTHER SUBSIDIES, BENEFITS AND SERVICES) Act 2016 aims to provide for, as a good governance, efficient, transparent, and targeted delivery of subsidies, benefits and services. While the objectives for the Act are noble, the manner of implementation has been shoddy and it often accomplishes the opposite of what the Act attempts to do.

Much of the debate around Aadhaar and UIDAI often finds the participants occupying extreme ends of the spectrum, calling either for its dismantling it or a flat out refusal to discuss its flaws. This post is an attempt to find a middle ground and discuss concrete steps that can potentially weed out some of the flaws and work towards a more citizen-friendly version.

The Rationale

Systems can be used (independent of the design or intent) in ways that includes or excludes the participants. The primary objective of a welfare program is to ensure that the deserving must get their benefits and, secondarily, it should eliminate, fraud and wastage. A truly inclusive program will not deny the deserving, while working in parallel to eliminate the non-deserving. A program that excludes as a default will result in a lot of the deserving being excluded from the benefit in the pursuit of efficiency over fairness.

As it stands now, the social contract that Aadhaar sets up between the government and the citizens is one where the state considers the citizen an adversary looking to unfairly benefit off it, till proven otherwise, solely on the basis of the existence of an Aadhaar number. This stems from approach that Aadhaar is the solution to a multitude of problems.

Solving problems of identity, authentication, efficiency and transparency require multiple tools and not a single tool. Some of these problems are completely unrelated to the issue of identity. It is more than possible to have a population tick all the boxes to certify efficiency, identity and transparency and still have a lot of the problems remain unfixed even with 100% Aadhaar coverage.

The Crucial Design Flaw

The UIDAI projects the zero-knowledge nature of the core Aadhaar services as a key facet that allows it to guarantee a high level of privacy. This is absolutely correct if you look at it solely from the perspective of the services that UIDAI provides. The flaw is that a service that only authenticates identity cannot guarantee things like efficiency, good governance etc.

Aadhaar alone does nothing to enable this; it depends on the larger ecosystem to provide this, which may or may not be possible using Aadhaar. Thus, claim that Aadhaar in its current avatar will enable it all is incorrect.

Let us take an example of how this works in real life:

Sita is a bank account holder with Example Bank. Bank wants to authenticate if Sita is who she claims to be. They collect and send Sita’s Aadhaar number and verifies it with biometric or SMS OTP to validate that it is indeed Sita who the bank is interacting with.

This is all what Aadhaar provides within the realm of identifying someone. It does not tell the bank if Sita is a great person. It does not tell if Sita can be trusted to pay back her loans etc..

What happens after this between the Sita and Example Bank is of no concern to UIDAI. This design allows UIDAI to not have to deal with what happens further in the loop. They call it the “zero knowledge” interaction. But, at the same time, UIDAI claims that just by its existence the Aadhaar number makes things more efficient and transparent without having any interest or say in how that is being done.

This flaw is glaringly obvious when things do go wrong, as it has happened in the recent spate of data disclosures by service providers across the country. UIDAI washes their hands off these problems as not theirs to solve. The resulting scenario is one where there is a distinct lack of ownership of the data that is collected by service providers.

The service providers only know they need to collect the Aadhaar number of their users. UIDAI will have nothing to do with how the data is collected, how it is used and how it is disseminated. The beneficiaries are left in a situation where, should a problem arise with the seeding where it is mandatory, they risk losing access to services that they were previously legitimately entitled to with no clearly mentioned redressal mechanisms.

Another problem in taking this “Aadhaar will fix everything” approach is that it cannot fix everything.

Let us go back to Sita to see how this can play out.

Assuming Example Bank has already seeded Sita’s Aadhaar number, without any further authentication, they can still provide services in Sita’s name and it will look perfectly legitimate if the service requires only Aadhaar as the key identifier. This is precisely kind of fraud that is most prevalent in India and it is one that Aadhaar, with its zero knowledge model, cannot solve.

On the other hand, it is very much possible that during an audit where Sita’s use of the services are crosschecked with another data source and found to be fraudulent. Based on this alone Sita can be denied services for no fault of hers and there will be little recourse available for her to prove that this is not true.

Lastly, not having an Aadhaar number can result in the denial of a service that Sita is legitimately deserving of otherwise and probably has the documentation to support too. This fails all of the stated goals of Aadhaar and also fails the fairness principle. In effect, you suddenly denying people who are perfectly legitimate and should receive the benefit.

As it stands, the entire process of seeding and how that data is further used is flawed and a growing mess. It needs to be rethought and re-calibrated to address the crucial flaws. Some of the ways in which this can be done:

1. Halt the seeding until a framework is established to ensure all stakeholders are properly educated about how to handle Aadhaar data. Currently, very few people know the entire picture. Even fewer people know what is the correct picture.

The way in which it is being done right now carries the risk of putting the service provider DBs in a state from where it will be difficult to get a clear picture of the linkage between the internal ID and the Aadhaar number. This is a hard problem to solve and needs to be done in a measured manner; unlike the breakneck speed at which it is being attempted.

2. Stop denying services on the basis of no Aadhaar if there are other documents that provide identity. Even the Aadhaar Act itself stresses on the this aspect where people should not be denied access to services if they can prove who they are with other documents.

3. Appoint an external auditor for auditing both UIDAI and the service providers. The idea originally was proposed in this excellent technical paper on Aadhaar: http://www.cse.iitm.ac.in/~shwetaag/papers/aadhaar.pdf

4.  For the current service providers who have seeded with Aadhaar data, audit them to ensure compliance. A good place to start from would be to enforce the AUA checklist for service providers: https://authportal.uidai.gov.in/static/AUA%20Compliance%20Checklist.pdf

5. Set up comprehensive processes and documentation for service providers to ensure that seeding is handled properly. Make it mandatory for seeders to explain in clear terms what is being collected, what happens in the backend and how the data will be used.

6. Publish granular stats in a voluntary manner about state/district/taluk level about Aadhaar(enrolment, auth attempts, auth failures). If the government truly believes in data, then embrace it and use it to better services. Take the ambiguity out of understanding how well is it working. The data is already there: need to make this searchable easily visusalized.

This is a good start, but it needs a lot more of work: https://data.uidai.gov.in/uiddatacatalog/dataCatalogHome.do

7. Have a single location that lists information about valid service providers and the reasons why they are performing Aadhaar seeding. Provide a simple mechanism (email/SMS/toll free number) by which people can verify the details regarding a service provider who is performing the seeding.

8. Initializing seeding should be a clearly defined process. This should be done with a circular from the service provider first, followed by a verification message from the UIDAI that a particular entity has been cleared as a valid seeding partner.

9. Provide a simple means by which an Aadhaar registration center/AUA/ASA can be looked up and validated.

10. Establish guidelines for making Aadhaar mandatory for something. You should not be able to randomly point at something and say, make Aadhaar mandatory for it. Establish SLAs for a region or a service before it can be brought under consideration as a mandatory thing.

11. Introduce penalties for SLAs not being met in Aadhaar-mandatory environments. Make it non-mandatory till the SLAs are not met again.

12. Ensure that service providers are given service specific tokens than just yes/no. Make it illegal to store Aadhaar numbers in any manner outside the CIDR. Ensure they are periodically audited for compliance in storage and dissemination.

13. Educate the customer how enrollment is done, de-duplication is done, auth is done.

14. Educate the customer about their rights. How to ask questions, what to ask questions about. They are completely in the dark right now. So much so that the likelihood that anyone asking for Aadhaar right now will get it easily from the people because they are used to giving it out for everything.

15. Educate the user about consent: at enrollment, authentication.

16. Make the user part of the de-duplication process. There is no visibility the user has in this at the moment.

17. Make the Aadhaar documentation better. There are numerous formats/versions floating around. There is no consistent way of versioning docs, naming them or having a clear location from where you search them, access them or find versions properly.

18. Don’t lose control of domains (uidai.net that used to host services is no longer with UIDAI).

19. Have a comprehensive redressal framework in place.

Edit: Added one crucial point from @kshashi:

20.  Be more open to criticism and researchers studying the underlying technical and policy framework. There are a good bunch of people who work on the right side of the law researching these things. Have them on your side working with you.


Now that I am officially off the gig that I was busy with the last year and a half, I have been pondering what is to come next? There are a couple of ideas that I have been playing with and a few interesting conversations, but I figured it would be a good idea to write down the broad contours of how I would like to work in the years I have left.

  1. Ethics: In the world where growth-at-any-cost is the popular, investor-friendly option, ethics often go for a toss. Doing the correct and decent thing is not often highly rated. I am actively trying to stay out of environments like these. The environment produced by such companies are toxic and it has an adverse impact on everything important to me.
  2. Remote-friendly: We are in an age where we at least the newer companies should actively try to shut down daily commute to the workplace. You need not create a workplace that is purely virtual. You can easily keep an office, but you can require your employees to come in only on select days when everyone is guaranteed to be in the office. Otherwise, build a culture that encourages and thrives on remote work.

    Of course, this is not going to be easy and culturally it is nearly-impossible to retrofit this once a non-remote culture has really set in. But the modern commute really has to die or reduce drastically. It makes our cities crowded, more polluted, adds stress that produces nothing additional and cuts time that can easily be used for more positive things.

  3. Good people: Find smart, good people to work with at all levels. From co-founders to co-workers it is important find the right kind of people to work with. It is worth it investing in both finding and training the right sort of people to form the core of the company. The right ones tend to stay longer with you, work better and keeps everyone happier.
  4. Make it better: For the customers you serve, the employees you work with and the industry you work in. This results in happier people and better output all around.
  5. Work is not everything: Allow people to explore other things when time permits. If times does not permit (for years in a row), you are not planning it right. 

Change The Contract

The contract between the state and its citizens is one that is primarily punitive in India. It promotes (and thrives) on fear and attempts to use that fear towards getting people to comply.

Consequently, the average citizen is not invested in helping the state prosper as there is no correlation between the well-being of the state and its citizens. The well-being of the state is considered just an unavoidable cost of living in the country.

A state that is not in the service of its citizens has its citizens preferring to avoid any contact with it (police, courts, offices), if they can help it. The exceptions to this are few. Nearly all interactions are driven from desperation; or are interactions led by the state.

What makes it worse is that the state does not incentivize good behavior. As a bad person, as long as you don’t get caught, you are much better off than the people who behave well. Meanwhile, doing the straight and narrow does not get you anything better than the ones who don’t do the same. In a lot of cases it gets you far less than the ones who don’t go by the book.

This, in turn, incentivizes people towards cutting corners and finding workarounds; turning a whole lot of the population into reluctant crooks. Income tax is a major example of this as for those who pay their taxes honestly, there is an increased risk of scrutiny. On the other hand, the risk:reward ratio makes it worth it for the privileged as the rewards are much higher for them than the less-privileged to dodge taxes.

Incentivizing good behavior is not an approach without its flaws. Every system will be gamed, especially in a country like India. But a gamed system that doesn’t benefit the deserving is far more flawed, compared to a system that benefits the vast majority of people who do the right thing while some still game the system.

If India has to stand any chance to transform itself into a developed country the contract between the government and its citizens has to change. The citizens need to stop fearing the state and know that the state has their welfare as the most important thing.

Anything else — war-on-corruption, cashless economy, poverty alleviation etc. — are small ideas that doesn’t address the core issue. They are just nice themes to rally around, while the rot at the core remains the same.

A New Domain

Most of 2016 has been a blur, due to varying levels of chaos on both the personal and professional front. The strange coincidence has been that healthcare and various aspects related to it has featured prominently on both fronts.

Healthcare, on the work front, has been a discovery. It is a complex, slow-moving beast with so many different angles to it. The technology itself is fascinating. Some of it predates even early versions of what power the modern web. Encoding formats that are separated by pipes and information systems that are still, by default, operate in silos and contained within facilities.

The technology aspect alone is massive opportunity. It is not an easy opportunity because of numerous entrenched players and, healthcare, being a necessity, is ridiculously prone to opaque pricing and cost structures. But, even the smallest positive change that can be made in the domain will touch tens of thousands of lives for the better and that is an attraction like nothing else.

Every aspect of health — from wellness to allergies to chronic conditions — technology can play a major part in improving how we live and how we can fight diseases. The enabling of it all, though, is a different ballgame. Things take years to come together at a basic level and it is a highly-regulated industry.

Which go on to show the size of the opportunity again; that even with such high levels of regulation, the lopsided demand-supply equation ensures that the sector remains a big draw for anyone looking to make quick money by providing magical cures and quick fixes.

The road ahead is going to be a long and arduous one.

There And Back Again; To Windows

After three long years with the Acer Aspire V5-431 it was time to get a new laptop for me. The Acer was a wonderful companion and it has travelled everywhere with me and survived everything from 50 degrees Celsius to -23 degree Celsius, plenty of dust, moisture and whatnot.

It was also starting to show its age and with a processor that was really under-powered, it was starting to get in the way of work. Additionally, it had a severe limitation of being only a 32-bit laptop, even though the CPU was 64-bit. Acer, in all their wisdom, decided this was a perfectly brilliant thing to do and made life miserable for people like me.

The age of 32-bit computers are rapidly coming to an end. Google has pulled support for Chrome on 32-bit for Linux a while ago. With the state of security online these days, running an unmaintained browser is begging for trouble.

Additionally, there is a host of software these days (Docker is a prime example), that don’t have officially supported 32-bit versions. It is far easier to move to a proper 64-bit computing environment than lose your hair trying to get oddball 32-bit ports of these things working.

The replacement laptop is an Acer again, this time the Aspire V13 (V3-372T-5051). It is a nice lightweight laptop with a great display, good battery life, 256 GB SSD drive and has the ability to take a maximum of 16 GB of RAM. That last detail is very important for couple of reasons as we’ll get to see soon.

Back To Windows

The laptop came pre-installed with Windows 10. For the past three years, I have been using the really nice ElementaryOS and had been quite happy with it. My usage of a computer in the past 3-years has also changed considerably to far lesser development work and more of text and various other kinds of documents. This meant that Linux was not really required for me as must-have to do development work and the little that I had seen of Windows 10, I quite liked it. It has the right amount of bling without going overboard with it as OS X has done in recent years.

While the idea was to always dual boot and use Linux as my primary device, having used only Windows in the past couple of days, I have come around to the idea of using only Windows. For most of my development needs, there is Vagrant and Docker (which is where the 16 GB RAM will come in quite handy) and other things that I am tinkering with on tech (Rust and Go) have officially-supported Windows builds.

About Unicorns And Related Things

Monday morning brings with it a post laying out the new ‘strangeness’ in the universe of the Indian unicorns by my former boss, Haresh Chawla. Some of the points, especially the one on doing due diligence, are ones that have bothered me for a while. I mean, even as an outsider, without access to the P&L statements of the companies it is not too difficult to mock up a hypothetical model of the business the investors are putting money into. So, why is it that the landscape seems to be littered with what seems to be strange moves by all parts of the ecosystem, unicorn or otherwise?

Be warned, what is to follow is a mix of personal experience, anecdotal information and lots of conjecture.

The crux of my argument is based on the factors that play an important role in the India story.


This is the ability of a team, enabled with capital, to execute a product/platform. It means that things work as advertised, as a norm. Whatever does not work is quickly fixed and the state of the system is such that edge-cases are minimal.

In some markets, this ability is determined by the team’s chops at quickly setting up a product operations, in other markets, it means primarily how enabled is the team towards closing key alliances, leads and relationships. In India this factor is particularly important because who you know can make a significant difference in being able to close a deal than the outright finesse of the product of the state of the finances of the start-up.

Market Size Validation

The truly gifted salesman can sell an ice cream to an Eskimo, but not even the most gifted salesman cannot build a big business selling ice creams to Eskimos. Every early-stage business and market makes assumptions about the size of the market it is selling to. The gap between starting a business and the business hitting escape velocity is often marked by the time spent in validating the market size and the long-term margins that can be accrued in selling to that market.

Market Growth Potential

Some markets start small, but given the right conditions, it can grow into substantial ones. A classic case of this is mobiles in India. 15-years-ago, this is a market that barely existed. Today, it is a multi-billion dollar industry. Given the right conditions, can a market grow like this?

Path To Opening Up Market

A market that is large enough or one that has enough potential to grow a lot alone does not mean that you can open up that market. Some will require a lot of capital to acquire users (deep discounting and CoD that did this for Indian e-commerce), others will require regulatory roadblocks to be lifted. There are various paths to getting this done, some are feasible, others are not.

Ease Of Access To Capital

Every business needs money to run and most early stage businesses will always spend more than what they earn. Which means that they need money in the bank to cover for expenses till the tide turns and they can turn at least cashflow positive. Easy access to capital means a lot of undeserving ideas/companies will also get funded, but a lack of it means a lot of good ideas/companies will never get funded.

Operational Efficiency

Operational metrics is key to measuring the health of a company. A company that does not manage to make at least a couple of operational metrics more efficient as the years go by is a big stonking red flag that everyone needs to take notice of.  If you keep needing more and more to do less and less and at some stage the ability to acquire the more is going to run out.

Capital Efficiency

This one is self-explanatory and the classic “how much do you make on your dollar?” question.

Exits, M&A

Investors and founders (especially the former), need an eventual big payday for all the risk and effort they have put into the start-up. A healthy ecosystem needs a regular supply of exits (through IPOs, M&A) for the payoff and also to correct over-leveraged players. Not all exits are of the sexy kind, where founders and investors make lots of money. But even fire sales are necessary to let the early risk takers cap losses. An ecosystem where exits are far and few in-between will struggle to sustain itself in the long run.

The Indian ecosystem, seen through the prism of the above factors, has gone through two cycles so far. For the sake of convenience I will ignore the smaller cycles (the 2008 meltdown, for instance).

 The First Stage (1997 – 2005)

  • Execution was terrible
  • Market size unknown
  • Market growth abysmal
  • Path to opening up market unknown
  • Lack of easy growth capital
  • Operationally inefficient
  • Capital inefficient
  • Unit Economics Is Bad
  • Very few exits, M&A

The Second Stage (2005 – 2015)

  • Execution has improved significantly
  • Market size has been validated
  • Market growth has picked up
  • Path to opening up market is known
  • Plenty of easy growth capital
  • Operational inefficiencies have skyrocketed
  • Capital inefficiency has skyrocketed
  • Unit economics has worsened
  • Exits and M&A has picked up

The key to unlocking the value in the Indian ecosystem is to get all the points to go green. We have improved significantly in execution, validating the size of the market and figuring out how to open up that market. But the two key factors — of improving operational and capital efficiency — are key to the long term well-being of the ecosystem and we are far from being able to crack open those two fronts. It is imperative that we figure out how to do that in the next stage at least.

Through 2015, I had the opportunity to see up close some of the e-commerce operations struggle with extreme inefficiencies. These operations can easily improve margins significantly if they can reduce inefficiencies. But that is easier said than done.

Some of the reasons behind the inefficiencies:

A System That Monetizes Inefficiency

Our famous ‘jugaad’ system ensures that people who can navigate around that can quickly acquire wealth and a significant number of people benefit from the existence of that inefficiency. A component of an ecosystem that brings itself into play in the ecosystem because of its complexity will always strive to increase the complexity in the system as a matter of survival.

Most of the e-commerce companies deal with problems on a daily basis that are in place because of this inefficiency. We have varying tax codes, local laws and levies. Till we get in place an administration that has the political willpower to remove these inefficiencies, it is anyone’s guess when that point in time will arrive when the people who benefit most from the inefficiency will stand to benefit more from an efficient and predictable system.

No Standardization

You would assume that after so many years into the e-commerce revolution in India, there would be a standardized framework of addressing pin codes in India. The funny fact is that there is not one. There are numerous ways of doing this and some of the courier companies even make up their own pin codes. Even where the pin codes are the same, the areas they consider under coverage of a pin code can vary from company to company. There is no standardization on buckets of weights or measurements either.

All this leads to an environment where an apple is not the same apple for everyone. An apple can be various kinds of apples and each shipment of apple takes a conversation with all stakeholders which results in disagreements, disputes etc.

No Experience In Large Scale Retail

The scale of e-commerce possible in India requires prior experience at the scale of what retail is like in North America, where it has been refined to a fine art with decades of experience driving logistics, pricing and marketing. The largest Indian operation on that front, which comes closest is a Big Bazaar, which is tiny compared to the large American retailing operations.

The lack of this experience has resulted in a bonanza for the same sellers across the platforms, while for the customers there is not much in terms of differentiation other than price. Our discounting is also not well thought through, because we don’t understand discounting as well as we should.

All this results in the current strangeness of the Indian unicorn ecosystem. From the investor side, most of the money being put into the market is just what the investors have to put in during each round to stay in the game. There is little to guide them, even in these times, to show what is possibly a good bet. The market is such that someone can easily outspend the top player into a position of dominance as there are really no moats that cannot be overcome with money.

This also leads to the overemphasis on founding teams than pure product in India. A good team in India that can move the wheels faster is always likely to win more than a great product with an inexperienced/not-so-well-connected team.

Set A Different public_path for Laravel applications on Webfaction

In a previous post, we had covered the issue how to get a Laravel application running on Webfaction.

While the approach worked well enough, the idea of symlinking to the ‘public’ directory was not a good one as every newly created file would have to be symlinked to make it available in the ‘webapps/appname’ folder. An Envoy task could easily accomplish this, but it just won’t scale on any website that has user uploaded files that has to be shown on the site.

Thankfully, Laravel provides an easy fix to this, which is to use the IoC container to bind the value of of public_path() to a different location. Since we don’t have this problem on the local setup, it is also a good idea to throw in environment detection into the mix.

All you have to do is open your bindings.php file (saved as app/bindings.php and included from app/start/global.php) and add the following lines:


if (\App::environment(‘production’))
\App::bind(‘path.public’, function()
return ‘/home/{{username}}/webapps/{{appname}}’;



Do Yourself A Favour; Don’t Secure That Server

Shellshock should come as a timely reminder to every person who is connected directly or indirectly to a device that is accessible from the public internet. And that footprint will most likely cover almost every living person in the world today, though most of us won’t have the faintest idea how it affects all of us.

But, the point of this post is not to educate you about the vulnerability. The point of this post is that even I can’t, at this time, say with any degree of certainty that I have a good handle on the attack surface of the problem on the servers that I run. I have been dealing with servers for close to 15-years now and used to run my own stack for a long time, but I no longer do that.

Keeping even a single public-facing server secure these days is not a simple job. It is not good enough to be even somewhat well-versed in security issues anymore. Security is a full-time job and if you are not a specialist, you are exposing your business and your users to risk that you don’t have any idea of.

Incidents like the story of Codespaces serve as a frightful introduction to how badly all of this can go wrong. Even so, I don’t see much improvement in the approach most companies take towards the issue of security, trying to cut corners by not involving specialists for the job.

Every couple of weeks, I am asked to help out a start-up or a big company who has some problem or the other in their infrastructure. A good number of these cases, it is a given that the security is either lax or non-existent. A recent case involved a start-up that had zero instrumentation in place other than the high-level overview that the AWS reporting interface provides.

The great thing about the cloud revolution is that it has made access to quality infrastructure and the cost of that access easy and ridiculously cheap. I can spin up a server, on a public IP with 16GB RAM, in a minute. Which also means that an idiot like me can get root on a fairly powerful machine without having enough knowledge or ability to properly secure it.

That ease, increasingly, is becoming a huge problem.

While it is true that everyone has to start learning somewhere before they get really good at anything, the fact is that this growing universe of servers that are not properly secured now represents a nice pool for the numerous number of bad actors who thrive on these things. As a result, I gave up managing servers on my own a while ago (the incredibly complex iptables rules on a well-secured sever quickly demonstrated how out of my depth I was on the topic) and don’t recommend self-managed services to clients either.

Even so, in most cases, clients wind up going the AWS route, trying to handle the serves on their own without having a good enough team in place to secure them or they try to cut corners in the worst possible manner and go for the cheapest deal out there. In either case, a couple of months down the line they are hit with performance or security issues that they can’t fathom. And in the worst case scenario, their servers are broken into and they have no idea that they have been compromised.

By the time the realization happens that something is wrong, reputations (in the case of data theft) are destroyed or the ability to grow the business is curtailed due to infrastructure issues. In trying to save the market salary of one good IT professional, organizations wind up giving up many times that number in terms of lost revenue.

If you are a decision maker in any organization, I urge you to get in place some professional help sooner than later. This is a problem that will only grow as more and more devices become accessible from the internet. Don’t wait for that big break in before scampering around to fix the problem.

Deploying A Laravel App to Webfaction Using BitBucket

It is not uncommon to use a shared host to run a Laravel application and I have been doing that for a while on my preferred choice of shared host, Webfaction. This post will help you do the same without having to modify much about your application.

The first problem you will face is that default PHP on Webfaction is still 5.2.x. This is not a problem when you are using the webapp itself, as the .htaccess file that is used by Webfaction will ensure that you use PHP 5.5.x, if you chose that as the app profile when you set it up.

The problem arises when you have to run artisan commands either directly or as scripts from Composer. These will pick up the default PHP in /usr/local/bin/php than /usr/local/bin/php55, which is the correct one for our purposes.

To get around this you need to do a couple of things.

First is to symlink /usr/local/bin/php55 to /home/username/bin/php

Second is to download a copy of composer to /home/username/bin/composer

Once these two steps are done, add /home/username/bin to your path.

Now we will install the Laravel app. I usually choose the location for this as /home/username/app since /home/username/webapp is controlled by Webfaction’s setup.

This post won’t cover how to set up a Bitbucket Git repo. There are plenty of great resources that will allow you to that, but I will make the following assumptions:

1. Git will be set up to be accessed using SSH on Bitbucket.

2. You have set-up key-based SSH access on your Webfaction account to Bitbucket.

3. Your Laravel application has been properly pushed into the repo.

These are the last few steps to be followed:

1. Clone your Laravel app into /home/username/apps/appname.

git clone git@bitbucket.org:xxxx/xxxx.git /home/username/apps/appname

2. Run Composer update to pull in all the dependencies.

/home/username/bin/php /home/username/bin/composer update

3. Remove the .htaccess file in /home/username/webapps/webapp_name/

4. Symlink the public folder to the webapp folder.

ln -s /home/username/apps/appname/public/* /home/username/webapps/webapp_name/

5. Edit the .htaccess file in the public folder to add the following lines:

Action php55-cgi /php55.cgi
<FilesMatch \.php$>
SetHandler php55-cgi

You are pretty much good to go after that. One thing you need to keep in mind that if you check in other folders or files into the public folder, you will have to symlink the new files.

For a much more convenient two-step deployment, you can use Envoy to manage everything on the server.

My workflow is mostly:

1. Push code into Bitbucket.

2. Run an Envoy task that pulls the changes into the server, runs composer dump-autoload and gets going from there.

Big Move Of The Year: Migrating Team-BHP To E2E Networks

It is always a great feeling to bring together two good organizations together and that has certainly been the case for me with Team-BHP and E2E Networks. Team-BHP is arguably one of the biggest automobile forums on the internet, run by a bunch of really passionate petrolheads, with an audience that even with a restricted membership is massive by any standards. E2E is the spiffy young hosting provider on the block, especially for sites that have a big chunk of their traffic originating from India, run by a bunch of geeks who make everyone’s life much easier by knowing, better than anyone else, what they are talking about when it comes to hosting infrastructure.

The Background

I had already done two projects with Team-BHP, both dealing with custom development of certain new features on the site. They have been one of the best clients I have worked with, being meticulous in what they do and most importantly, they know exactly what they want to do get done, which makes a vendor/consultant’s job a breeze, compared to the usual one-line brief that we often get to work with. The site had been hosted with WiredTree since 2009 and it had largely been a good experience, but things had slipped in recent times with the growing requirements, so they were on the lookout for a new home, which was preferably in India as the majority of the traffic for the site originates from here.

Decisions about infrastructure at this scale is never straight forward. There are numerous factors to be taken into consideration, some of which are:

  1. Level of support (managed or un-managed?)
  2. Cost of bandwidth
  3. Hardware SLAs.
  4. Application-level support.
  5. Traffic mix (is it equally geographically spread or largely local?)
  6. Connectivity at the service provider’s end.

Taking an informed decision about this is often not possible for most organizations as it requires both experience and knowledge that is usually quite specialized and not readily available within organizations. This a the crucial gap bridged by a consultant like me. Moreover, a property that is the size of a Team-BHP would normally have been in operation for at least 4-5 years, meaning the application stack often has a lot of legacy issues and complicated dependencies to handle. They have to be taken into account and best practices have to be rolled out where it is possible, without disrupting existing operations.

The Big Move

After evaluating various options, we decided to go with E2E Networks. They were already doing managed services for some substantial online properties from India and were also hosting some of the properties of the companies I knew personally and the feedback had never been anything short of spectacular. I am also very partial towards top management in infrastructure providers who are reachable on the phone and know what exactly are they doing and Tarun from E2E is a prime example of that. What helped was also the fact that they went out of their way earlier, before signing up with E2E, to fix a major problem that was bringing down the site while on WiredTree. It is not often that you get to see something like that in the industry.

Finally, after much testing and some delays (caused by the rather ill-timed Heartbleed bug and a DNS reflection attack), we started moving the sites late this week and the final piece of the puzzle (the main forum) was moved to E2E today. As things stand they look quite good and stable and hopefully both Team-BHP and E2E Networks will have a long and fruitful association.

Why Not AWS?

An important question in this regard that I often face is, “why not AWS and a CDN?”. For one, AWS is not cheap, especially when you have to get decent, hands-off managed support for it. Insecure public-facing web servers are the bane of the cloud hosting world and unfortunately a staggering number of companies learn about this the wrong (and often costly manner) way in the end. A poorly secured box with multiple cores, 10+ GB of RAM and a 100 Mbit unrestricted port is any hacker’s wet dream. And there are just way too many of them out there.

In the case of a CDN, it is not a panacea for site delivery. The effectiveness of a CDN depends on a variety of factors. For one, other than Akamai and Bitgravity, the other CDNs don’t have POPs in India. Which means that your Indian traffic will be served by routing it out of the country. Secondly, they are quite expensive and don’t make sense till you push incredible amounts of traffic, which not many sites actually do. One of the reasons why we chose E2E was that they had decent peering (via Netmagic) to all the major networks in India, which made it a faster option compared to most CDNs.

Edit: As pointed out by Manu J on Twitter both Cachefly and Cloudfront now have POPs in India.

The Ideal Infrastructure: Seamless Develop, Test & Depoly

In today’s world, most parts of developing, testing and deploying a website can be automated in a cost-effective manner. While the initial process to get this in place can be complicated and time-consuming, the long-term benefits of having this in place will save any organization time and money in the long run. Done right, this can also be aligned well with an organization’s business objectives. While it used to be really costly and difficult to accomplish this seamless process even five years ago, it is no longer that hard or expensive anymore.

If you want to explore rolling this out in your organization, do get in touch and I’d be more than happy to help you out.