The 4 levels of training for software engineers in Silicon Valley

I'm a young, educated, white guy. I'm very, very lucky. And life is unfair. So if life is unfair - working hard is not the point at all, knowing how the world works is the point.

What this self-made tycoon said before the age of 23 is not wrong, just as a long article originally written by a Chinese netizen, "It is difficult for a poor family to produce a noble son", said:

"Efforts need to have a direction and guidance. If they are guided by the wrong mindset, your hard work may only earn you a meager income to make ends meet."

Yes! Efforts need to have a direction (the point of this article is really not to tell you to find a rich second generation or a rich man to marry/marry ...

When it comes to engineers, you might think that anyone who is a geek and can write programs is a programmer. In fact, after working here for a few years, I found that in Silicon Valley, software engineers can be divided into the following levels:

Can write some code

I will repair the corpse written by others.

Can do some software architecture, can lead people, and finally

Invested in the development of a large Apache Project.

In Silicon Valley, the abilities and salaries of these levels of engineers vary greatly, but the biggest difference is actually in opportunities. Many programmers who don't understand these levels will spend their entire lives wandering between the first and second types of engineers. Although they work very hard and try very hard to learn new technologies, they often feel that time is running out for them. When they get older, they can't compete with the young guys and are naturally eliminated. This gives outsiders the wrong impression that engineers never age.

So once you step into the field of software engineer, you can only grit your teeth and fight your way through each level one by one. This is the great journey you must face.

Can write some code

Although many people would say that studying computer science is useless, or that people who are not majoring in computer science can also write programs, etc., in Silicon Valley, studying computer science is still more popular. Otherwise, you can go to Facebook to find people's pages, the Software Engineering section, and see how many job openings do not require a BS or MS degree in Computer Science?

(The interesting thing is that 99% of the top companies here don’t just want to recruit people with a master’s degree, which is very different from those fake big companies in Taiwan)

Or maybe you have heard that you can start programming halfway through, for example, by taking a General Assembly course and getting a high salary right out of it. I haven't met anyone who has started programming halfway through and got a high salary, but as far as I know, if you don't have a computer science background and have no previous programming experience, you usually need to do an "unpaid internship" for a company to accumulate market experience and trust before you can officially become a full-time software engineer.

Although Silicon Valley companies are relatively indifferent to academic qualifications, they still care about your background before you have experience or a portfolio to prove yourself. On the contrary, after working for three to five years, if you have no experience but only your background, you will gradually be looked down upon by others.

There is also a very direct way to confirm whether you are a real software engineer. You can find a local headhunter and ask him if he is willing to recommend your resume for you. If he is willing, then you are 100% sure.

No matter how you became a software engineer, in short, your job involves writing code, so that counts as a software engineer.

Can debug other people's code

Generally speaking, if you can modify other people's corpses (badly written programs, but not including homework XD), you are probably qualified to be called a senior engineer.

Writing code is easy, modifying your own code is easy, but modifying/correcting other people's code is actually not easy, especially the kind of code that doesn't have any comments at all. You not only have to be familiar with the language and technology he uses, but also understand his thinking logic and blind spots, which is much more difficult than doing it yourself.

With the current scale of software projects, you have to work with others. You can certainly ask the person who wrote the code to maintain the code, but what if that person gets sick/takes a leave/quits? The product and project still have to go on, so the ability to "repair the corpse written by others" becomes very important.

Basically, when you reach this level, your LinkedIn inbox will gradually be filled with messages from headhunters who have job vacancies and hope to interview you. As long as you are familiar with the data structure and algorithms you learned in school and have a good grasp of the interview atmosphere, it should not be a problem to negotiate an annual salary of more than US$100,000.

Can do some software architecture and can lead people

Any software engineer with a little sense knows the difference between "making Amazon" and "making the Amazon website", and this difference is the biggest factor that defines this level of software engineer.

(The Amazon mentioned here refers to the Amazon shopping website, not AWS)

The biggest difference between “making Amazon” and “making the Amazon website” is the scale of the service. The technology and algorithms required for 200 users and 20 million users are completely different.

Take the simple function of the product recommendation system as an example. Let’s assume that you have your own recommendation algorithm. When a customer logs in to the website, you must use the recommendation algorithm to calculate all your inventory items to recommend products that the customer may like, thereby increasing the revenue of the shopping website.

Suppose you currently have 10,000 products. When you have 200 users, you only need to count 2 million times, and a better machine may be able to handle it. But when you have 20 million customers, you need to count at least 20 million x 10,000 = 200,000,000,000 times, not including the calculations required for the subsequent sorting. How would you design this system?

Suppose you use 100 machines to process, and some of them are miscalculated, what will you do? Will your results be affected? Also, you certainly can't calculate it on the spot when the customer logs in, because it will definitely be too late, so how do you store these results that are recalculated every night? If Amazon wants to reduce the time it takes to generate a web page to a few milliseconds, you can only store these results in memory instead of on the hard disk, so how do you design and configure this large memory system (assuming we use memcached)?

So far, it is still just a "product recommendation system", not to mention other functions, database design, distributed computing planning, etc. What's more, the amount of data and customers of Amazon are much larger than this!

In addition to designing and implementing these software architectures, this type of software engineer must be able to debate with a group of people, and after passing the questions, tests and suggestions of many engineers, lead the team to produce the final version of the product, and maintain and improve it.

Common titles for this type of engineers include Software Architect, Director of Engineering, VP of Engineering, and CTO.

In fact, the biggest difficulty in becoming this kind of engineer lies not in the technology, but in the first-hand practical experience of designing and implementing these large-scale systems. After having the experience of architecting this level of service, future owners or teams will believe that you have such ability and hand over more and larger projects to you. Only then can you make a living by doing so and establish your own brand in the industry.

Among the engineers I have come into contact with, there are two ways to upgrade:

If you have worked in a similar department in a large service company, such as Amazon, then it is easy to find a leadership role in a competitor or startup with relevant experience in the industry leader.

Worked in a fast-growing startup, where I helped build or maintain this level of services and products.

Invested in large-scale Apache Project development

I know you long for Guido van Rossum who designed Python, or for someone who can hatch Linux at home like Linus Torvalds when you have nothing to do, but I don't want to discuss these god-level language creators, because their experiences and abilities are a bit difficult to replicate. Instead, I want to discuss their followers, the huge group of Open Source community engineers who quietly push Linux and Python into the future.

Have you ever heard of the secret unions in Silicon Valley? The author is right, I am talking about them in this category.

(If you haven’t read the article yet, go check it out first)

Many engineers who are passionate about technology will download the source code of these technologies to study them, and will also interact with the Open Source engineers who contributed to the technology to exchange technology. After a long time, whether they are itching to do something or have been waiting for a long time for a certain feature, they will roll up their sleeves and start writing it themselves.

Although theoretically anyone can work on an Open Source project, it takes a lot of effort to get everyone to agree to formally include your program code in the project.

First, let’s assume that you have added some features to the project. You will go to Github to create a Pull Request. Next, these veteran engineers will start to comment on your code.

"Why do you write it like this? It doesn't fit in with the structure of our original code."

(Boss, I'm new here. How would I know your original architecture?)

"Why don't you use this component and do it yourself?"

(Because no one told me before XD)

"Your test unit runs too slowly the way it is written. Try another way!"

(So ​​that means you want me to take it apart and rebuild it, right? XD)

… Although you may feel that this is very strange, think about it, where can you find so many senior software engineers to take the time to review your code? This is a priceless opportunity, and now you can get it completely for free. You should have treated them to a meal a long time ago. XD

After going through the above hardships, your program code is finally accepted and becomes part of the Open Source project, and you are upgraded to an Open Source contributor (Contributor/Developer).

If you want to do Open Source, it is recommended to start with popular Apache projects, such as Spark or Hadoop. There are two main reasons:

Because the project is popular, there are many senior engineers who have contributed to it. The quality and quantity of their work review will be much better than other Open Source projects. The chances of enlightenment will increase greatly, and you can practice faster.

Once you become a regular contributor to these popular projects, or even selected as a committer by the project elders (see here for the classification), your reputation in the industry will naturally rise. According to the irresponsible report of my friend, when many super companies are looking for people, they pull out the list of developers/committers of related Apache projects and call them from the first to the last. Generally speaking, the annual salary of a committer of a popular Apache project is at least 200,000 US dollars.

Of course, if the entire Apache project was initiated by you, your value would be the same as what is said in this article about unequal pay for equal work, and there is no way to measure it with common sense.

The most terrifying thing is that if these Jedi Knights also have entrepreneurial spirit, the companies they create will make you feel like the sky is the limit. If you think Cloudera (Hadoop) is too old, although its market capitalization is really amazing, you can take a look at Databricks (spark), this 8-person company that was established only one year ago (2013) has attracted the most famous venture capital in Silicon Valley to rush and immediately invest 47 million US dollars (let's calculate how much each employee can get XD). Both companies were founded by such engineers.

By the way, many of these engineers like to mentor newcomers. As long as you don’t waste their time, they are willing to help you. If you meet one, seize the opportunity.

The channel discussed so far is what I have seen and heard in the past few years as an engineer in Silicon Valley. You are welcome to discuss with me. If you know a leader in other industries or professions who can provide more great channels, please write to me and tell me. I am very willing to talk to him and see if I can write down his experience and insights to share with everyone.

Link to this article: http://www.cocoachina.com/programmer/20140920/9697.html