Common misunderstandings about large software companies

(philipotoole.com)

47 points | by otoolep 5 days ago

15 comments

  • keyshapegeo99 1 hour ago
    Many of the author's rebuttals hinge on the assumption that everyone in an organisation is acting in its interest first - and not their own, often conflicting, self-interest. As such, they are not particularly convincing.

    Large organisations absolutely do, as a function of their scale, produce pockets where slackers and incompetents can hide. They'll surround themselves with a web of process, pointless meetings, and substance-free buzzword-heavy documentation/presentations to disguise this fact. Others may become ensnared in this web, and will rightly express the criticisms that the author is attempting to debunk.

    • rob_c 56 minutes ago
      I think this is akin to x% of the worker ants doing all the work. Once you get to a big enough scale and have to delegate I'm sure every company hits this.

      I just wish we didn't have to rely on hiring 100 on paper workers for 5 excellent people committed to the company...

  • aleph_minus_one 1 hour ago
    > “There are too many meetings” At very large software companies, programming ability, technical expertise, and raw resources are not the limiting factors. Coordination is.

    In my opinion there exist much more efficient ways for coordination: for example, write down some really good documentation and explanations that are then read by the other stakeholders, so that these, at the end, also have a very deep knowledge about the topic.

    Nearly all employees have studied at a university, so the people are very used to writing texts (papers, seminar papers, lecture notes, thesis, ...).

    In my experience the reason for too many meetings is rather that many managers love meetings.

    --

    > “There is too much process and bureaucracy” [...] At a very large software company, the software matters. It may be relied on by millions of people. It may underpin businesses, infrastructure, or daily life. It may not be particularly glamorous software but it has to work. It has to keep working. Failure is not charming, and recovery is not always cheap. [...] Process exists to manage risk, correctness, and scale. Calling it “too much process” without acknowledging the stakes involved is like criticizing a bridge for having too many safety checks because you once built a treehouse with a hammer and some nails.

    This is one reason. Other common reasons for so much process and bureaucracy are

    - Many managers love processes, because they can "hide" their failures behind processes, and introducing new processes and bureaucracy lets the manager pretend that he is doing something to solve the problems that plague the department.

    - Many processes and bureaucracy are simply demanded by the legislature when you work in some heavily regulated industry. These legal demands often don't make sense.

    • andrewflnr 1 hour ago
      > Nearly all employees have studied at a university, so the people are very used to writing texts (papers, seminar papers, lecture notes, thesis, ...).

      I wish. Most people I've known in universities seem to read and write the absolute minimum to get by.

      But I tend to agree that writing is preferable to meetings in most cases. I want to try out a policy that all meetings of more than two people must produce a written artifact, or clarifying edits to an existing document, that explains whatever ambiguity required a meeting to clear up. But you also need people to read. People don't read.

    • Etheryte 1 hour ago
      Just because written communication works well for you doesn't mean that it works for others nor that it's the best way to communicate about everything. There's a place and time for both. For example with documentation, it's nice to have accurate and well thought out docs so you can search and read through it, but oftentimes it's faster if your teammate just tells you what bit you need. Meetings are the same way. We've all been in meetings that could've been an email, but that doesn't mean every meeting can be an email.
    • AnimalMuppet 1 hour ago
      Meetings:

      People who can write clear, unambiguous, accurate technical documentation are relatively rare.

      And a meeting is in fact sometimes the best way of coordination. We have a question. We have five different people with relevant input into the question. Even if all five can write well (and will do so in a timely manner), we still need to reach a consensus on what the right answer is, and to make sure that everyone's issues are heard, and that they feel that they have been heard. A meeting often does that better than shooting emails back and forth among the five people.

      Processes:

      Processes are often added because something went wrong (often expensively wrong), and a process was created to make sure that it won't go wrong again. But what they miss is that the process also has a cost - a dollar cost, and a time cost.

      Worse, there's a limit to how many processes most people can remember. You can create a process, that's the 47th process that your people have to remember, and when they don't keep it because they don't remember it, you can blame them. So here I kind of agree with you.

      > Many processes and bureaucracy are simply demanded by the legislature when you work in some heavily regulated industry. These legal demands often don't make sense.

      Maybe they don't. Ignore legal requirements at your own peril, though - they can have some pretty nasty teeth.

  • zug_zug 30 minutes ago
    I've spent 20+ years in the industry seeking to understand first, and my conclusions have pretty consistently been that the systems are broken and inefficient as people already talk about. The caveat I'd add is that the critics themselves would also be just as much as a crapshoot.

    Honestly it sounds to me like the author doesn't truly understand the inherent conflicts of interest at a large company. For example a really common anti-pattern is "Nobody knows X thing is a problem my team manages in a problem (e.g. our app eats battery usage), but if I draw attention to it they'll want to measure it forever. So do not make it a focus."

    In short pretty much never does any employees/manager's/executives interest align with the company.

  • alphazard 2 hours ago
    Equating process to risk aversion is a mistake I hadn't seen before. It's true that moving slow makes it less likely to make something worse per unit time, but it also makes it less likely that anything will get better. It means that you don't try many things, and can't get important things done quickly.

    You can ensure quality by making features opt-in, having a beta program available to risk tolerant users, adding QA resources, having representative users in captivity (employed at the company).

    There is no law that says that you must move slow or do less in order to be low risk, you can also do a lot, move fast, and only let the best out.

  • jackconsidine 1 hour ago
    This important devil's advocate perspective reminds me of Chesterton's Fence [0].

    I used to run a dev shop and had the opportunity to work with companies of all shapes and sizes. The startups often discovered Chesterton's Fence by declaring they didn't need this or that (meetings, accountability measures, etc), only to learn the hard way why they existed.

    And meetings, beauracracy, et al are rightfully criticized for being inefficient and fostering mediocrity. But I think I'd agree with the author that it's glib to say meetings are dumb, no need for hierarchy without understanding their purpose

    [0] https://en.wikipedia.org/wiki/G._K._Chesterton#Chesterton's_...

  • ttoinou 2 hours ago

       Coordination is almost free in a ten-person startup. It is still relatively easy in a forty-person company. 
    
    
    I find coordination difficult even for two / three persons for any given topic where there the tree of dependencies (of sub tasks or others topics related to it) isn’t trivial and there are unknowns to research. Unless those persons are doing the same thing and are constantly communicating, which is very expensive
  • andrewflnr 1 hour ago
    Someone who says there are too many meetings is probably actually saying they are having bad meetings. If they got value from those meetings, they wouldn't be complaining. So there is likely still a problem to address.

    Also, as a somewhat trivial side note, an instinctive reaction to not getting the clarity you need from a meeting is to ask for another meeting. So even if the optimal level of meetings is annoyingly high, bad meetings will probably push the level of (bad) meetings even higher. So you'll still actually have "too many" meetings.

  • hn_throwaway_99 2 hours ago
    I liked this post. I may have some minor qualms (e.g. while I think execs should be proxies for the customer, they have many other competing motivations that can push customer needs way down), but I especially liked the closing section:

    > Understanding before criticizing

    > Large software companies have real problems. Some are structural. Some are cultural. Many are self-inflicted. But many of the behaviors people complain about are not pathologies – they are consequences.

    > If you want to criticize how large organizations operate, it helps to first understand why they operate that way. Without that understanding, the criticism may feel sharp, but it will not be useful.

    I see that kind of "criticizing before understanding" all the time on HN, and while that's probably just inherent to an open forum, commenters who do that should realize it makes them come across as "less than insightful", to put it generously. Like I see tons of comments often about how managers only get to their position through obsequious political plots. And sure, that may exist in some orgs. But you can always tell when folks have never even considered the competing forces that act on managers (i.e not just the folks they directly manage, but requirements coming from higher ups, and peer teams, and somehow being responsible for a ton when you actually have few direct levers to push) and solely view things through the lens of someone being managed.

  • kbos87 1 hour ago
    As a technology company scales up, making great software becomes one of a hundred things the company needs to do right in order to survive and grow. Doesn’t mean it isn’t absolutely essential, but so is having a strong GTM machine, finance competency, operational rigor, HR, and a long list of other essential functions.

    It’s only the tech industry where the voice and ego of small companies hold outsized share of voice and love to claim the contrary.

    • aleph_minus_one 1 hour ago
      > It’s only the tech industry where the voice and ego of small companies hold outsized share of voice and love to claim the contrary.

      I'd be a little bit careful with this claim:

      The fact that small companies can have such opinionated opinions without going bust is to me a sign that in particular for software development (but I don't claim that this is transferable to other industries) small teams/companies do have an efficiency advantage.

      Many hypotheses can be formulated why this might be the case, like

      - software industry is less regulated

      - writing good software as the company's product requires a lot less collaboration between many stakeholders than what is necessary for producing other types of sellable products

      - in software, "having a smart, though opinionated idea" is of a much bigger advantage (also for the company) than in other, more established industries

      - ...

  • zahlman 1 hour ago
    None of this seems particularly revelatory to me. It comes across more as just an argument against letting software companies get large.
  • mslt 2 hours ago
    One nitpicky detail is that the executives may be a rep for the customer/consumer, but are also very much reps for the shareholders and that’s a pretty big distinction
  • readthenotes1 2 hours ago
    I prefer Parkinson's take on the matter. Parkinson's law has a lot more to do with why companies grow so large. Why WhatsApp can can't write meaningful software with fewer than 10 people
  • msla 2 hours ago
    I did not expect to see big-company apologia on Hacker News.

    https://en.wikipedia.org/wiki/Apologia

    The thing this (very good) post doesn't mention is that big companies select for blub languages because that's where the most low-cost labor is, in that you can hire multiple Java developers for the cost of one Haskell developer, even if Haskell might be objectively a better choice for the project.

    • Etheryte 2 hours ago
      I don't think this is a charitable interpretation. As a business, you need to be able to backfill positions or hire more when the need arises. If you use a language that's very commonly used, it's a lot easier to hire. There isn't anything sinister to that, it's simply reasonable.
      • anonymars 1 hour ago
        There was a post, I think on the Uber engineering blog, that stuck with me. It essentially boiled down to: it's easier to change the tech stack than the hiring pool, and talked about deliberately setting something up that was technically less optimal but easier to hire for

        Corollary: it's perhaps easier to throw money at fancier hardware to improve performance, than the alternatives

  • mji 26 minutes ago
    > At very large software companies, programming ability, technical expertise, and raw resources are not the limiting factors. Coordination is.

    If coordination is your limiting factor I'd argue that it shouldn't be, and you're not investing enough in removing it as a factor. Companies can use various tools to do this, for example:

    * Defining directly responsible individuals / single-threaded leaders so that every choice doesn't involve massive coordination

    * Putting people who work together in the office sitting next to each other most days of the week

    * Having a strong culture of async communication that is visible to the broader group by default, for example with Slack

    • joshuamorton 10 minutes ago
      > * Defining directly responsible individuals / single-threaded leaders so that every choice doesn't involve massive coordination

      Obviously doing this makes sense when possible, but in cases where you see it, it's usually due to a fair amount of work to have made this possible.

      To give a simple example, I work on a class of problems that require a VP level approval for new instances of $thing. I've gotten this down to a pretty straightforward process where I can work with folks who are proposing a new instance, get things working, and then the VP usually is happy to stamp the work we've already done. Though in some cases they aren't, and they have (good, probing) questions or changes they'd like us to make. But that's only possible because I have a longstanding relationship with the set of folks who ultimately approve this kind of thing, I have worked with them on a process that they like, and I have the experience to shepherd things effectively, and their trust and buy-in. And I'd argue that my case is pretty simple, because while there are a handful of responsible people that I could go to, there's rarely any individual concerned.

      Consider a different, relatively common case, of a client and a server that are owned by different teams. The client wants a feature in the server, and perhaps is willing to do some development and loan headcount to implement the feature. Who is the single responsible party here?

      It should probably be either the manager of the client server team or the mutual manager of both if the teams are fairly close in the wider org, but then you have multiple indirect layers between the people doing the work (client team members) and the accountable person (server team manager), and that's the state you get to after you've done a bunch of coordination work and gotten everyone to agree to that division of labor and accountability.