Magento – Love You, Love You Not – Untold Saga Of 8 Years EMA

Hey friends, this post is about my experiences with Magento – the world’s most popular open-source e-Commerce platform since OsCommerce (a very popular Open Source e-Commerce platform of 1990s and early 2000s)

I receive several queries from my professional network asking for my views on Magento.  The queries are usually related to usability, adaptability, performance and scalability of Magento in various application contexts.  So I have finally thought of penning my views down to avoid repetition as well as share my knowledge with a larger community.

Why are my views on Magento important?

Lot of people seek my views on Magento despite of the fact that there are millions of interesting articles about Magento already floating in WWW.  Why are my views important?  Because I am the first adopter of Magento in India – yes the first one!

Magento debuted for general public in March 31, 2008.  But the majority of development of first release was done in 2007.  Varien, the founding company of Magento (ex OsCommerce guys) released their first beta on August 31, 2007.  I, around the same time in 2007, was scouting for a reliable alternative to OsCommerce (another very popular open-source e-Commerce project) that I had been working for about 6 years already.  My intentions were to launch my own e-Commerce startup in India (RateTag.com); and I already knew that OsCommerce was not the right choice (it lacked Object Oriented programming style of PHP5 and there were several gaps in functionality which I would have to bridge by way of 3rd party modules or my own work) for the next 10 years.  During this search, I discovered Magento – they were at 0.9 beta version then.  I was simply floored by the amazing functionalities these guys had built in the beta version itself.  It was very easy for founders of Magento to create it because they had great lessons learn during their time in OsCommerce; that’s why instead of forking OsCommerce, they decided to do new project completely ground-up.  I immediately downloaded the beta version, ignored the beta warning saying it’s not production ready yet, and launched my e-Commerce play RateTag.com on 30th September 2007 (one month after the beta release).  That’s how I became the first adopter of Magento in India.

Since my adoption of the beta version, I continued to upgrade my local copy of software with each subsequent version irrespective of whether it was a major or a minor release – It did not matter to me – I had a craving for all the features they used to add or optimize with each release.   In-fact, I upgraded so frequently, that I had to write a script that used to poll their SVN repository daily for a new commit and upgrade my local copy automatically.   Therefore I have had a fare share of my personal highs and lows (successes and failures) tightly coupled with the amazing history of Magento.  I almost had an extra-marital affair (EMA) with Magento because I was spending more time with it than my lovely caring wife.

I am just one of thousands who use(d) Magento.  However divide the user base by two for a moment.  First half will represent those gray haired folks (or already bald like me) who migrated their old online business from OsCommerce or other PHP based carts to Magento and the second half will represent those black or golden haired folks (or blondes) who used Magento as part of their first e-Commerce venture.  I am lucky that I am part of both these halves; and hence my experience with Magento is quite unique.

Therefore I consider myself reasonably competent to comment on Magento – my views should help a Magento user (existing/potential) take a right decision.

Magento – Why I Love You?

1. It’s Versatile – Magento is a well thought e-Commerce product overloaded with functionalities (aka features).  The manufacturer ships this vehicle fully loaded so you rarely need an accessory.  This has been a culture and they have always maintained it well.  Each version creates curiosity and strong craving to upgrade immediately.
2. It’s Well Networked – The Magento community is a hypermart in its own right of e-Commerce accessories and tools.  In this community you can find solutions to virtually any kind of feature you may need or problem you may face.   The Magento community is enormous and global.  You will find here people from all over the world contributing their plugins and snippets to make life easier for others.  This is a big reason why Magento is such a huge hit – entire world adopted it very fast.
3. It’s Accommodating – Almost every opensource project has a challenge in accommodating an extra-large community.  Magento has been very successful in this because of its modular design.  Almost every solution contributed by community comes in form of a plug-in that you can simply deploy from a web based admin interface or shell command.
4. It’s Validated – Magento has already proven it’s worth in some highly coveted e-Commerce ventures.  Several well know global brands use it for their online stores or catalogs.  So there are ample case studies to wow and get inspired from.
5. It’s Growing – Magento has been growing and it’s not slowing down anywhere.  They recently released Version 2.0 which is a major overhaul of the platform from 1.x versions.  They way Version 2.0 looks, it shows a road-map of another 10 years for the company.
6. It’s Easy To Modify – Magento has been written in PHP using Zend PHP Framework.  It uses Object Oriented Programming paradigm in PHP.  PHP has been traditionally an easy language to learn; combine ease with Zend F/W and OOPS, your life as a Magento coder is much easy.  Moreover you can quickly code and refresh your browser to see the results without having to go through the pain of compilation and packaging.
7.  It’s Easy To Extend – OOPS and Modularity of Magento platforms makes it every easy to extend/override existing functionality or write new ones by simply using ‘copy-paste-rename-mod’ strategy on existing modules using latter as a template or blueprint for similar activity.
8. There Are No Credible Open-Source Alternatives – Honestly there are no alternatives as beautiful and awesome (assets-wise) as Magento.  I have worked on OpenCart, PrestaShop, OsCommerce, OfBiz, X-Cart, Zen Cart, Drupal, VirtueMart (Joomla Plugin) – no one is even close to Magento in terms of features, modularity or size of community.  No other opensource e-Commerce project as on date, to my best knowledge, can boast of a road-map that could potentially cross Magento’s road-map in next 5 to 10 years.
9. It’s 100% Free – The community (opensource) edition of Magento has been 100% free for use.  They have always given full access to the community version and whatever features they have in the community version, they are all accessible from single installation.  And whatever features are there in the OSS Magento (plus contributions from community) are generally sufficient to launch an e-Commerce play with very little investment in coding.  I have tried other e-Commerce OSS products; while they easily lure you from their bold marketing punch lines, however once you spend some time playing with them, you realize that they have locked key features you can’t go live without.
10. It’s Very Web 2.0 – Magento uses lot of Ajax and Fancy stuff on front-end.  It uses Prototype library for java-scripting.  One can also use Jquery along with it in compatibility mode.  Then it has a very well written templating framework which can be tweaked to render almost any kind of UI without having to touch to core application service classes. Therefore you will find that almost all Magento driven sites are quite stylish (very Web 2.0)

Magento – Why I Love You Not?

1. It’s Overwhelming – Magento is a behemoth.  It’s a giant e-Commerce product that comes as a single orderable SKU in terms of functionalities.  Therefore there are hundreds of features in it which a typical e-Commerce play will never require.  But you do not have an option to exclude what you don’t need at the time of ordering.  You get a truck when you can manage with a bicycle.  Hence you need equally big a server (or few) and technical resource (or few) to use and maintain it.  I would even dare to estimate that 80% of Magento users never use more than 20% of the features.
2. It’s Monolithic – In a world of Microservices and Agile Development, Magento is a misfit.  It is a monolithic web application that ships and work as a single tarball of zillions of lines of code.  Therefore it is very difficult to split the code apart, focus on areas of interest (concerns) (think Microservices paradigm as an alternative) and develop or enhance product in an Agile Distributed development environment with hybrid set of internal and external development resources.
3. It’s Not Cloud Friendly – Magento, they way it is pre-packaged, is not cloud friendly.  A cloud environment (AWS, Azure or Google Cloud) is an elastic pool of virtualized computing infrastructure with several attachments like memory, storage, network etc. (think of host instances running of Xen hypervisor virtualization fabric).  The clouds I just named are public places – they restrict users on each resource available – Memory, CPU, IOPS, INodes, Traffic, Storage – everything is rationed and billable on pay as you go basis.  Therefore, if you simply download a copy of Magento, upload it on a LAMP (Linux Apache MySQL/MariaDB PHP) server/container on one of the cloud compute instances, it will simply not work – I am 100% sure about that.  You will have to seek an expert’s help to optimize your cloud containers (whole cloud configuration in-fact) and configuration of Magento as per the design of your business (your context).  Someone may argue that there are pre-optimized ready to deploy instances of Magento for cloud – I challenge that such readymade instances (think AMIs in AWS) are mere pieces of demo and without a considerable re-configuration/fine-tuning effort by an expert, they are useless for any real-life business use.
4. It Doesn’t Believe In Sharing – OsCommerce was popular because people could deploy it on small Virtual Private Servers (VPS) which used to cost them $10 or so per month at the web host.   VPS (servers) are virtual instances of host operating systems running on shared hardware.  The administrator of visualization system controls how much compute resources are to be given to each instance.  Pretty much the same way cloud instances work today.  Before VPS became popular, OSCommerce used to run like a wind on shared hosting where the host used to give login access to a particular folder (Doc Root) on a physical server for uploading PHP files.  The site used to run off a virtual host configured in Apache web server using the designated folder of user as DocRoot.   Magento doesn’t believe in all this.  When Magento was adopted in 2008/2009, people assumed that Magento will work on their existing VPS plans or may be a higher one.  They never knew that dedicated servers (notice plural) will be required to run Magento.
5. It’s Slow – Yes, it (Magento Community Edition) is as slow as a Tortoise.  The Enterprise version is faster but it is very expensive – not everyone can afford it.  Magento’s design through the version 1.X age was never focused on speed.  The company focused on features only.  The net result is that the Magento CE  versions are utterly slow and unsuitable for ‘traffic heavy’ websites.  The Magento EE is faster than CE versions but it is still slower than what is required in today’s competitive world.   There are several reasons why Magento is slow.  I will only touch upon some key pointers for sake of brevity.
   • The community version (1.x series) does not have any in-built caching mechanism to store and deliver frequently used web objects.  The enterprise version (1.x series) uses Zend Framework’s Container Cache services to render frequently accessed web objects – this adds dependency on Zend Containers (extra cost) and is still not the best practice caching implementation.  Version 2.X which is just launched in November 2015 has full-page caching in both Community and Enterprise versions but-but-but this version is bleeding edge and is yet to be market validated.  The migration of 1.X to 2.X is not straightforward click-of-a-button affair.  It is being said in community that industry will stick on to version 1.X for another 2 years.
   • Magento uses SQL Database as its datastore.  Majority of people use MySQL with Magento.  Magento uses Zend ORM framework to talk to SQL database.  The SQL queries of Magento are not tuned for speed and performance from SQL point of view.  This has a direct impact on user experience.  In absence of a well thought database fine-tuning, Magento (CE or EE) will under-perform.
   • Magento web frontends are heavy-weight lifters – there are too many XML layout files and HTML snippets that work together to render each page of Magento.  They are utterly sluggish if you ask them to run.
   • Magento is heavy code everywhere – thousands of intertwined PHP classes, bootstrap, warm-up and chained executions make it an inefficient code in terms of speed of execution.
   • Magento does not leverage threading/parallelism paradigm – every execution of Magento code is a uni-threaded sequential execution of a long chain of PHP classes which further depend on a chain of Zend framework classes to achieve their objectives.  It does not leverage threaded models that are possible with JAVA or .NET type of languages.  This limitations comes from PHP and will stay there for a long time.
6. It Does Not Upgrade Seamlessly – Of hundred of upgrades I did in my copies of Magento, I am unable to recall a single instance where the upgrade was as simple as a click-of-a-button.  Every time I upgraded, the SQL upgrade commands will fail at one place or another either because they tried to modify database to such an extent that the connection timed out or there was a conflict or a SQL exception.   On top of this if, I was using 3rd party plug-ins, it was 100% guaranteed that an upgrade will bomb.
7. Sluggish Checkout, Broken Funnel – Magento uses bad SQL language in checkout portions.  This is apparent when multiple customers are on checkout at the same time and are switching between the checkout tabs (login, shipping, payment, preview) etc.  The blocking SQL commands lock SQL tables which makes concurrent checkouts almost impossible.  Due to this checkout part of my funnel always used to give me nightmares.  Whenever I used to pump-in money on bumping up the traffic, my conversions simply did not add-up to the kind of spend I did.
8. It’s Fuel Guzzler – Magento is a a fuel-guzzler in all ways.  It requires a shop owner to spend exorbitant amount of money on hosting, developers, system administrators and designers.  To be honest SMEs don’t have that kind of budget.  So they must rely on time-sharing consulting services of freelancers which is a pain in itself (freelancers are ad-hoc, moody, generally busy, disappear after some time etc. etc.).  The evolved customers (heavy transaction load) can afford to pay for the resources to manage their Magento deployments but they will never be satisfied with the performance of Magento speed-wise or funnel-wise.
9. Enterprise Community is Expensive Affair – The Magento Enterprise development community is a different club all-together.  There are very evolved consulting companies of repute and talent who are Magento Certified partners.   So they should be able to sail you through your bad times with Magento – however, the costs are exorbitantly high.  I remember sending RFPs to few Magento partners for a greenfield Magento project for an Enterprise grade user I was working for.  Both of them almost unequivocally set a minimum budget guarantee of USD 1,50,000 before they even opened my RFP PDF file 🙂 .  On top I was looking at spending about USD 30,000 on Magento EE licenses and another USD 1,25,000 on first year of production support.  Worst part was that the Non-Functional Requirement of maximum 10 seconds of page rendering time was not acceptable to any of these vendors at any cost.
10. Magento changes too frequently – That’s an asset and a liability.  For me it was more a liability than an asset.  With each change came features I could not resist myself to adopt.  And with each release came implicit upgrading and performance issues.  Therefore it was very difficult for me to stick to one major release for at-least a year and tell it to myself that I will not touch my code for next 12 months.
11. APIs are as sluggish as the web front-end – The Magento APIs (PHP or SOAP Web Services or REST API) are as slow as the web front-end because they use the same services and DAO (Data Access Objects) infrastructure of the code that is used to drive the web side of it.  In my application, I had to aggregate product and inventory feed from multiple suppliers for which integration via PHP API or Web Services was key to success.  When I leverage these APIs, it took ages for an update of even 10,000 products to complete.  The code used to work at a speed of a clock – one product per second sorts.  Hence I had to (and I could manage to) hack my way down to low level SQL queries which did the trick for me.  I am sure Magento will never recommend this (your support will be void in EE version if you tinker with internals of the product) and you may not be able to even manage it.
12. Mobile App on Magento APIs – Forget it – see reason above – you simply can’t launch a responsive Mobile App if your are not sure of how well your REST APIs will work.
13. If you are hopeful of doing thousands of orders per day, Magento is not right for you – be it Community Version or Enterprise version.  A contrary view may be that there are so many success stories out there.  My counter argument is that technology has to be nimble, efficient and cost-effective.   Magento is not cost-effective at a scale of thousands of transactions a day.  It will remain an expensive product for many more years.  For people who operate in range of 500 to 2000 orders per day – a heavily optimized Magento deployment may work with a considerable cost overhead on COGS due to at-least one in-house Magento resource.  For people operating in range of 10-500 orders a day, moderately optimized Magento deployment can be sustained with help of a freelance consultant.   In any of the cases I just mentioned, the business owners will continue to heavily doubt the funnel performance of Magento – Web Analytics will show that quite clearly.
14. Does not leverage Content Delivery Networks (CDN) and Edge Delivery paradigms to the fullest.  It merely uses CDN for offloading delivery of static objects like images, css, js etc.  CDN is a very large discussion area – so I will be concise.  CDNs are used to deliver content to customer faster from nearby servers.  It can be used to deliver both static and dynamic content.  Magento does not integrate seamlessly with any CDN to deliver dynamic content.  It has to be manually modded to reap all benefits of a CDN.
15. Lacks Search Facility – User driven search is one of the key functionalities required in any online catalog.  If your system does not throw relevant search results, your customers will bounce off and go elsewhere.  Magento Community version never had a robust search technology.  It uses SQL based fulltext search queries (or simple like queries) on an table of indexed keywords.  This is not a true text search engine – rather a cheap substitute.  Therefore one would have to rely on 3rd party Solr/Lucene/Elasticsearch plugin to get true search capabilities.  These external modules are usually paid and they do not leverage full capabilities of modern search engines like Solr and Elasticsearch (think facets, aggregations etc.).  This is a very big drawback in Magento 1.X series.  The Magento 2.X series talks about in-built Solr index in EE.  The community version is still devoid of the true search capabilities.
16. Extremely Dependent on Indexes – Since Magento uses ORM framework of Zend to talk to SQL databases, lot of queries were optimized over a period of time by adding indexes.  Therefore there are indexes as patch for problems and there are indexes built-in as part of past lessons learnt by the company.  The net result is that a Magento store relies heavily on SQL indexes to work properly.  It does this as a batch which you would typically do at night when there is least amount of load.  In my cases the indexing process went on for hours.  In-fact I remember times when indexing process started at 1 AM and finished around 11 AM during peek load times.  Magento never spent time on an efficient incremental indexing mechanism which could potentially run the whole day rather than recreating whole index as a big batch.  They did some optimizations in the 1.7X version upwards but I had to keep automatic indexing completely off except for a couple of critical ones like stock and price.  There are mods available in community that tweaks indexing process but none of them are reliable.

Why Did I Sustain This EMA for 8 Years?

The only reason I could survive a 8 years EMA with Magento was because:

1. I was full-time employed with another company and my e-Commerce play on top of Magento was a personal strategic decision of acquiring knowledge of a fast evolving phenomenon.
2. I happen be a very competent technologist to handle a behemoth like Magento.  Since I am proficient in both development and system administration, I could manage the code in both lab and production environments for my use.  Majority of businessmen can’t do this.
3. I had high hopes from each subsequent release – since I was not relying upon Magento for my daily bread & butter, I waited for it to mature on its own till the time I needed it fully.   So we got along well with each other for good 8 years.

What Happened After 8 Years EMA?

After 8 years of fruitless affair with Magento darling, we parted our ways.  It was me who said bye bye.  I decided to spin-off my own e-Commerce play with some friends, hence, I needed a true e-Commerce workhorse that Magento wasn’t.  So rather that getting into another EMA, I preferred to help myself.  Today I am a proud owner of a rock-solid e-Commerce platform built ground-up with my knowledge of the e-Commerce business and technology.   However instead of running an e-Commerce play of my own, I have decided to focus on technology and product and let my software be used by e-Commerce startups as a ready launchpad.

About Myself

I am an entrepreneurial-spirited technologist who has managed multitude of business complexities by strategical implementation of technology in a career spanning over 20 years. I have worked in reputed online, software consulting, and startups in leadership positions.  A major portion of my career has been spent in e-Commerce industry.  I have worked in almost all departments of an e-Commerce enterprise.   I am also curator of a ‘best practice based’ high-performance hybrid marketplace technology which I give out to e-Commerce startups as a ready launchpad.

You can check my profile on LinkedIn : https://in.linkedin.com/in/akhildelhi

I will love to connect with you.

Your comments on this post are heartily welcome.