Databricks after Google and Facebook: How do they stack up for an employee?

I’ve worked at Google (Mountain View and Pittsburgh), Facebook, IBM TJ Watson Research Center, and Xerox PARC. But I’ve enjoyed my day-to-day the most at Databricks, a late-stage startup where I work and just finished exactly 5 months today. This might be surprising to those who consider these other companies their ‘dream companies’. So I’ve written a short note describing the factors that make me feel this way. But first, a bit of caveat.

A bit of caveat

This isn’t a scientific comparison by any means. It’s one person’s opinion across companies where the role/seniority weren’t the same, and the times of employment were also different. For example, I only worked as a junior engineer at Google, and only ran a large team (>200) at Facebook. Also, Google today is quite different a company compared to what it was in 2010. So these types of notes are anecdotal. Having said that, there are company-specific cultural idiosyncrasies that are consistent over time and across roles (e.g. the feeling of prestige); likewise, there are cultural properties that are driven by company-agnostic parameters (e.g. company size).

The things I like more at Databricks

Relatively speaking, I find Databricks to be more attractive to Google (henceforth just G) and Facebook (henceforth just F) for a few reasons:

1) Size: Databricks is an order of magnitude smaller than Google and Facebook. This has huge implications, such as being able to understand a big chunk of the company in its totality, e.g. here’s the engineering team, here are the set of things they are building, these are the leaders, these are the products, features, services, and so on.

• At G & F: Most people didn’t know other people and their projects even within a domain. It gave the feeling of being lost in a big city, being a tiny cog in a wheel, and a loss of identity. Those like me who came from academia, where were previously well-recognized in academic circles, upon arrival we became one-of-many contributors to massive projects.

2) Relative surface area: I am sure there is a technical term for this, but this is the concept of one’s ownership of the product or service offering relative to the whole company’s most important offerings. For me, Databricks offered significant relative surface area from the very beginning.

• At G & F: At both companies, I had small relative surface areas to begin with, but with focused effort and a series of early success, it grew more significant. But it could’ve easily gone the other way. They have so many products, few folks had large relative ownership, and when they did they had overlaps, e.g. a top researcher at Google Brain doing important work could be unknowingly overlapping with someone at DeepMind.

3) Talent density: My sense is that the average tech talent density is highest at Databricks, thanks to a very selective hiring process, and (via a feedback loop) the subsequent leveling up of everyone in the presence of high talent density. And thus far no crazy hiring sprints that invariably cause a drop.

• At G & F: At the very beginning of my tenure at G, the company was small and the growth rate was low. That time, talent density felt very high. But ever since, both G and F have had crazy hiring sprints, during which talent density inevitably went down. And this is a vicious cycle because those who are hired end up influencing future hiring decisions.

4) Process and Execution speed: It’s easier to get things done at smaller companies with less process, and Databricks is no exception. Decision making has less overhead, allowing the teams to move faster. The company is very mindful of the flip side. Hence, on average, a lot gets done.

• At G & F: A function of size and wild success, as companies grow, they introduce more process to try and preserve their success hubs with such vigor that they make it difficult to innovate and move fast. When they realize this, they introduce some types of incubation hubs, but those hubs struggle to succeed, and when times get tough, those hubs are the first ones to get axed. I saw this pattern at both companies. At G, most search quality launches took several months.

5) Transparent communication: People frequently share their opinion about what’s working and what’s not; they do this because Databricks feels like a safe space to do so without fear of rebuke. Leaders also generally share their opinions rather transparently, and this starts at the very top.

• At G & F: During my early days at G, it did feel like communication was very transparent, and it felt that leaders deeply cared about the employees and its culture of transparency. Over time, leaders became far more careful with every word they used, for fear of being taken out of context and turned into news headlines somewhere. Basically, most leaders started playing it safe.

6) CEO, CTO, Co-founders: We have a CEO Ali Ghodsi who is respected for being a straight shooter calling a spade a spade, being accessible to employees, and he still writes code! CTO Matei Zaharia, the creator of Apache Spark, is humble, accessible, and whose business sense is as exceptional as his technical sense. Likewise, all the cofounders are smart and approachable. And because most came from academia (UC Berkely, mainly), they bring in a certain degree of academic rigor to their thought process and decision making, which diffuses into company culture.

• At G & F: In my entire tenure at G and F, I only had one meeting in which Larry Page was present. I never met Sundar. The top company leaders rarely engaged with rank-and-file employees. Relatively speaking, Zuck came across as much more authentic, which was nice.

7) Product lines: Most Databricks products are useful for businesses; BS product lines are rare. Everything is driven by the market. They empower smaller companies to do warehousing, analytics, and AI that only much larger players have built in-house. So there are rarely any random features or functionalities, ensuring that most folks are doing meaningful work.

• At G & F: There were too many BS product lines that came and went at these companies. Even when some of them felt unanimously like bad ideas, they were pursued at the whim of certain leaders, only to be killed eventually. So many products felt dead-on-arrival.

8) External drama quotient: Because Databricks is not so much in the public limelight, and it’s primarily a B2B product, we are seldom distracted by external drama.

• At G & F: Both companies were always in the limelight, but F took the cake for being frequently in the limelight for all the wrong reasons that were huge distractions. Some of these events must have caused massive productivity to dip around them.

Things to watch out for at smaller companies

On the flip side, there are some things to watch out for when joining smaller and/or pre-IPO companies:

• Tooling tends to be less mature at smaller companies. e.g. IDEs, support for your favorite programming language, and so on.
• The equity compensation can be quite a bit at publicly traded companies like F and G, especially given how much those two have grown relative to the rest of the market. At private companies, you’ll almost certainly miss out on that for an unknown period of time.
• Getting hired is generally quite hard given the hiring bar + a much more limited absolute headcount growth compared to F and G.
• The peer group being of high talent density means that you might feel imposter syndrome sometimes. I definitely feel that way at times.
• Brand name recognition of G and F is higher than most out there let alone smaller enterprise companies. If that’s something you care about, you’ll be disappointed.
• At large companies like G, one can get away with slacking off for some time till they get noticed. Smaller companies cannot afford this.

A general note on the evolution of tech companies

Thinking about these differences for some time, a lot has to do with company size and the speed of growth. When companies have a healthy high margin business, i.e., a machinery has been set up against a dominant and widely used product, which is almost certainly true of Google’s Search Ads, for example, they start thinking about scaling. Scaling the usage of the product makes sense and is not controversial — you want your product to be used more and used world-wide, and there’s a tangible target there.

But scaling the employee base often puts company leaders in uncharted territory; how much to hire, how fast to hire, what kind of profiles, what work to make them do, and so on. And as much as they want their people growth to be balanced across their suite of products and services, the people slowly gravitate toward your shiniest, most talked about products. When they were a 1,000 people, the leaders talked about 3 top priorities; when they became a 100,000 person company, the leaders are still talking about 3 top priorities, even though now there is a long tail of less shiny products and services that someone needs to work on.

And that’s when culture starts to dwindle. Internal competition, scope creep, secrecy, etc. Couple that with a promotion criteria that emphasizes ‘what did this person do exactly’ or ‘how big was their team’, and the culture grows more toxic.

Is there a solution to this for companies growing larger? I am not sure. As companies grow, the properties and behaviors from when they were smaller no longer apply, leading to the emergence of a highly complex system that is beyond an individual’s ability to understand, let alone control. But AI systems can potentially make sense of complex systems via sheer brute force methods over historical data and present trends. Such systems might be able to inform company leaders how to grow optimally and in a way that can preserve culture to the greatest extent possible.

I’m trying 1:1 mentoring via MentorCruise. Feel free to sign up here.