Future of MongoDB: Fireside chat with MongoDB CTO Eliot Horowitz

Last night I attended a Meetup at MongoDB Inc.'s new Palo Alto office to hear MongoDB's CTO, Eliot Horowitz, speak about the product roadmap. With a new production release right around the corner and MongoDB World in the not-so-distant future, the buzz and excitement around all things MongoDB is high. For those who were not able to attend, we're going to recap all the major points Eliot made.

MongoDB 2.6

To start things off, Eliot announced some big news:  MongoDB will GA its next major production release (v2.6) in the next 4-6 weeks. This release includes extensive rewrites of MongoDB's internal architecture and exciting new features. We'll highlight some announcements here, you can find a complete list of improvements in the v2.6 release notes.

Query engine rewrite

The entire query engine has been rewritten, from the execution engine to cursor management. The motivation behind the rewrite was to accomplish a few goals: make the code easier to maintain, increase introspection into what the queries are doing and provide more insight into what indexes are used by queries and why.

Index intersection

As part of the query engine rewrite, a new feature, index intersection, has been released. MongoDB queries have been limited to only one index, regardless if it is a single field or compound index. With index intersection queries can utilize multiple indexes for performance optimization. For collections with many indexes e.g. upwards of 50, this feature aims to help reduce the number of redundant indexes.

Optimizations to the update() method

The update() method has been completely rewritten from scratch. The goal here was to write code that is easily understandable so that any future new MongoDB engineer can write a new update operator in a matter of days. Users should expect many more update operators to come.

Significant improvements to network architecture

In perhaps the most interesting segment of the night, Eliot quickly recounted the history of MongoDB and 10gen (now MongoDB Inc.). This history lesson was particularly important as it provided context for understanding why MongoDB was originally architected the way it was- MongoDB Inc. began as a DBaaS company.

To quote the linked post:

"When we first started 10gen in the fall of 2007, we set out to build a full platform as a service stack with MongoDB as the data layer. This was a fully hosted system (still open source), that encompassed a load balancer, auto scaling application server and data tier. The application side was a full server side JavaScript environment...

Writes in that system did not individually wait for a response from the database.  However, the application server itself always checked the database for any errors that occurred during the entire page load (using getLastError and getPrevError) so that the user/system would be notified of any issues... This worked great in the platform, as we were able to control the whole access pattern."

As a result, MongoDB has been working on a significant overhaul for networking around database operations. Operations such as inserts, updates, deletes, which used to require multiple packets now only use one packet, resulting in significantly less network overhead.

Unlimited result sizes for aggregation framework queries

When using the MongoDB aggregation framework on large data sets, it's possible for an interim result in the pipeline to exceed the size of RAM. Currently result sizes from the pipeline are limited to RAM; with the new release these queries are now no longer limited.

Looking beyond 2.6

Of course, after 2.6 is released MongoDB will continue to innovate. Eliot also shared his vision for some future roadmap plans.

Vertical scalability

MongoDB has a strong horizontal scaling story and is looking to improve its vertical scaling story. On the roadmap is to better CPU concurrency, index RAM consumption and spinning disk performance.

Replica sets

Unlike shards, which can run up to the thousands, replica sets are currently limited to twelve member nodes due to election logic. Large MongoDB deployments will be able to deploy more members in the future.

Along with support for additional members, MongoDB is looking to introduce more types of replica set members. For example, some member nodes may be solely used for analytics, not to be used to in production.

Got questions?

We look forward to hearing your thoughts and questions about the future of MongoDB. For specific questions, you can write to our team at any time and we'll be happy to discuss or help point you in the right direction.

18 Responses to Future of MongoDB: Fireside chat with MongoDB CTO Eliot Horowitz

  1. Nevi_me 2014/02/27 at 4:53 pm #

    I think what would ‘help’ the community post 2.6 GA is to lay out the plans for 2.8 and beyond, For people following the JIRA issue tracker, there’s been a number of “when is this coming” features that have been postponed, which happens in lots of projects, because one does not simply just write features. The communication could help ‘comfort’ people who are investing in MongoDB, both time, server set-ups and the support & enterprise revenue that we’re giving Mongo.

  2. Augusto Bernardo 2014/04/10 at 10:09 pm #

    When will MongoLab have 2.6 available on its plans?

  3. is motor club america a scam 2017/04/01 at 10:31 am #

    678239 387959Hello! Ive been reading your web site for a while now and finally got the courage to go ahead and give you a shout out from Kingwood Texas! Just wanted to say keep up the good work! 83254

  4. what is motor club america about 2017/04/02 at 12:55 am #

    302543 553337Register a domain, search for available domains, renew and transfer domains, and choose from a wide variety of domain extensions. 983653

  5. Best Best Online News in the World 2017/04/06 at 12:12 pm #

    552968 118461But wanna comment on couple of common points, The site style is perfect, the content material material is actually excellent : D. 850909

  6. Best Best Online News 2017/04/06 at 11:41 pm #

    94336 588983I think one of your ads triggered my internet browser to resize, you might want to put that on your blacklist. 44599

  7. Best Best Online News in the World 2017/04/07 at 5:08 pm #

    279983 803724Hey, you?re the goto expert. Thanks for haingng out here. 289649

  8. Best Best Online News in the World 2017/04/08 at 4:10 am #

    614593 66455I believe one of your commercials caused my internet browser to resize, you may well want to put that on your blacklist. 614318

  9. Best Best Online News in the World 2017/04/08 at 12:34 pm #

    32613 338878Ill create a hyperlink to the web page about my individual weblog. 637762

  10. GVK Biosciences 2017/04/09 at 2:58 am #

    29080 558686Hey, you used to write superb, but the last couple of posts have been kinda boringK I miss your super writings. Past couple of posts are just a bit out of track! come on! 874081

  11. Indian Escorts in Dubai 2017/04/19 at 1:11 am #

    206465 34373Really clean site , thanks for this post. 947855

  12. Switch Energy Supplier 2017/04/21 at 3:50 am #

    336519 184998Yours is really a prime example of informative writing. I think my students could learn a great deal from your writing style and your content. I might share this article with them. 857166

  13. Switch Energy Supplier 2017/04/21 at 2:37 pm #

    365724 591148I think this web site has very superb composed subject material articles . 672073

  14. Mark Hurd Salary 2017/04/26 at 9:50 pm #

    60078 629180Significant other, this superb site is fabolous, i merely adore it 504278

  15. canadian pharmacy online drugstore 2017/04/27 at 8:17 am #

    127380 183033I like this web site really significantly so significantly superb information . 508475

  16. Generico in farmacia 2017/04/27 at 10:16 pm #

    Online sito italiano, acquistare online con postepay e acquistare generico con paypal, acquistare generico online tranne prezzi del. Comprare online in italia anche acquisto on line pagamento alla consegna, acquisto in italia e comprare generico sicuro nonostante si puo comprare in farmacia

Trackbacks/Pingbacks

  1. MongoDB 2.6 – updated internal architecture and exciting new features, by Christopher Chen-Kung Chang – mor krispil - 2014/03/02

    […] MongoDB 2.6 – updated internal architecture and exciting new features, by Christopher Chen-Kun… […]

  2. DB Weekly No.3 | ENUE Blog - 2015/08/13

    […] The Future of MongoDB: A Fireside Chat with MongoDB CTO Eliot Horowitz […]

Leave a Reply