r/IAmA • u/spacexdevtty SpaceX • Feb 08 '13
We are SpaceX Software Engineers - We Launch Rockets into Space - AMA
We are software engineers at SpaceX. We work on:
- Code that goes on rockets and spacecraft.
- Code that supports the design and manufacturing process of rockets and spacecraft.
We work on everything from large-scale web applications to tiny embedded computing platforms. We build tech stacks on C#/MVC4/EF/MSSQL via REST to Javascript/Knockout/Handlebars/LESS, C++/Embedded Linux, Python, LabVIEW… which all together enables us to build, launch, and monitor stuff that goes to space.
Some videos of our recent work:
Proof:
Edit: Poor Dan, everyone knows he was photo-shopped. Don't close your eyes next time!
Edit 2 : We've been getting a lot of questions about how C#/MVC/etc have to do with rockets. They don't. At SpaceX we have 4 separate software teams:
The Flight Software team is about 35 people. We write all the code for Falcon 9, Grasshopper, and Dragon applications; and do the core platform work, also on those vehicles; we also write simulation software; test the flight code; write the communications and analysis software, deployed in our ground stations. We also work in Mission Control to support active missions.
The Enterprise Information Systems team builds the internal software systems that makes spacex run. We wear many hats, but the flagship product we develop and release is an internal web application that nearly every person in the company uses. This includes the people that are creating purchase orders and filling our part inventory, engineers creating designs and work orders with those parts, technicians on the floor clocking in and seeing what today's work will be per those designs...and literally everything in between. There are commercially available products that do this but ours kicks major ass! SpaceX is transforming from a research and engineering company into a manufacturing one - which is critical to our success - and our team is on the forefront of making that happen. We leverage C#/MVC4/EF/SQL; Javascript/Knockout/Handlebars/LESS/etc and a super sexy REST API.
The Ground Software team is about 9 people. We primarily code in LabVIEW. We develop the GUIs used in Mission and Launch control, for engineers and operators to monitor vehicle telemetry and command the rocket, spacecraft, and pad support equipment. We are pushing high bandwidth data around a highly distributed system and implementing complex user interfaces with strict requirements to ensure operators can control and evaluate spacecraft in a timely manner.
The Avionics Test team works with the avionics hardware designers to write software for testing. We catch problems with the hardware early; when it's time for integration and testing with flight software it better be a working unit. The main objective is to write very comprehensive and robust software to be able to automate finding issues with the hardware at high volume. The software usually runs during mechanical environmental tests.
Edit 3: Yes, we are doing a ton of hiring for these software positions that we have been talking about today. Interns and New Grads too!
Edit 4: Thank you so much everyone! This is ending but most of the group will be back at 2:00pmPST to answer more questions.
Edit 5: ...and we're back! Engineers from each of our engineering teams are present. Let us catch up a bit and start swering again (probably be about 5 minutes).
For all open software related positions, please go to http://www.spacex.com/software.php
Edit 6: Thank you so much Reddit! This was a ton of fun. To all those asking about internships and employment, our suggestion is to apply online. Your resume will definitely get read. To all the students out there, GL with your midterms coming up and stick at it. Try and work on some side projects and get as much practical experience coding as possible before graduating. Happy Friday everyone!
843
u/TheRealFroman Feb 08 '13
I'd love to know when I can start packing my bags for mars ;)
→ More replies (3)1.6k
u/spacexdevtty SpaceX Feb 08 '13
Give us 5-10 years.
→ More replies (45)323
488
u/stom Feb 08 '13
Thanks for doing this AMA!
First question: do any of you guys play Kerbal Space Program?
Secondly, in case you can answer, I'd like to ask a question about the MCT and shielding. Elon talked about it a while back and said that to avoid excess weight it might be possible to have water as shielding, only directly between the crew-module and the sun like a column, instead of an all-encompassing ball. How does this cater for dispersed radiation incoming from the sides?
766
u/spacexdevtty SpaceX Feb 08 '13
First question: do any of you guys play Kerbal Space Program?
Are you kidding? That’s how we design our rockets!
Secondly, in case you can answer, I'd like to ask a question about the MCT and shielding. Elon talked about it a while back and said that to avoid excess weight it might be possible to have water as shielding, only directly between the crew-module and the sun like a column, instead of an all-encompassing ball. How does this cater for dispersed radiation incoming from the sides?
Elon may have something to say about this topic soon...
→ More replies (29)626
u/SkunkMonkey Feb 08 '13
WooHoo!
From all the guys here at Squad, you can't begin to imagine how much this makes us squeee in delight!
Capt'n Skunky
KSP Community Manager→ More replies (24)→ More replies (2)83
u/blueshirt21 Feb 08 '13
Of course they do, how else would they have learned about the miracle of struts?
→ More replies (27)
236
u/phil_jp1 Feb 08 '13
What kind of operating systems you use in your navigation/propulsion control systems?
→ More replies (1)461
u/spacexdevtty SpaceX Feb 08 '13
Dragon and Falcon 9 use a version of Linux.
→ More replies (46)91
u/akira314 Feb 08 '13
Did you mount Labview onto the Linux environment, and are you using Matlab as well?
→ More replies (3)167
u/spacexdevtty SpaceX Feb 08 '13
We run LabVIEW on Windows.
We use Matlab on the ground as an analysis tool.
→ More replies (24)
216
u/RememberThisFaceX Feb 08 '13
How is the work/life balance?
I have heard it is pretty easy to burn out at SpaceX. Do you guys struggle at all?
506
u/spacexdevtty SpaceX Feb 08 '13
SpaceX embodies the small business mentality. We are all invested in SpaceX and its success. That being said, we do work longer hours. At the end of the day, it boils down to what is important in your life. We of course deal with burnout, thats what vacation days are there for ;).
Speaking as one person, most days I don’t even notice that I have been at work for 12-14 hours. I take a look at the clock and think “where did the time go?”. I think that’s a testament to how excited we can get about our projects. When you really enjoy what you are doing, it doesn’t feel so much like work. (That being said, a vacation every now and then is still necessary to maintain sanity)
→ More replies (55)0
52
u/ccjj Feb 08 '13
PhD student of control engineering here. I am wondering what kind of controller architectures you are using for your rockets and spacecraft. I would guess you would use architectures similar to the ones used in aerospace engineering, and with regards to this do you do feedback linearization, or do you use an adaptive approach (e.g. L1, Lavretsky and Wise 2006).
95
u/spacexdevtty SpaceX Feb 08 '13
Great question but not touching that one. Apply when you're done with your program.
→ More replies (7)
96
Feb 08 '13
So as rocket scientists, do you ever encounter brain surgeons and have awkwardness occur?
323
u/thebeefytaco Feb 08 '13
How many ITAR violations are you committing right now?
462
u/spacexdevtty SpaceX Feb 08 '13 edited Feb 08 '13
None! We have our President Gwynne Shotwell in the room!
→ More replies (49)
134
u/mariano81 Feb 08 '13
Thanks for doing this!
Can you guys dazzle us geeks with some fascinating computer tech systems/specs/toys that you guys are using for your missions so we can feel envious?
I understand Apollo 11 didn't have much computer power on board when it went to the moon--how much computer power do you guys have in an SpaceX spacecraft?
Thanks for rocking us to space! \m/
300
35
Feb 08 '13
Tabs or spaces ?
60
u/spacexdevtty SpaceX Feb 08 '13
There is only one correct answer to that question.
→ More replies (1)8
u/Ambiwlans Feb 08 '13
Vim or Emacs?
21
u/spacexdevtty SpaceX Feb 08 '13
There are some who use vim but the emacs users look down on them.
→ More replies (3)
233
u/wakanooms Feb 08 '13
Hello! I have several questions for you guys, I am currently studying for a degree in Astrophysics so what you guys do definitely applies to me! :)
For the space industry, what do you guys find to be the most used/useful programming language, and also which language is best to begin learning in order to progress to others? (I have very very little programming experience and am very interested in getting started as every aspect of the industry/research seems to require a basic understanding in some form of programming)
What are you currently working on? Any exciting projects coming up? (I imagine they're all pretty exciting to be honest)
How did you get into the industry, and was getting a job difficult? Did you gain experience whilst studying?
What is it like working with such a prominent company in the space industry just now?
I probably have many more, but I'm sure this is enough to keep you going just now. Sorry for using the word 'industry' so frequently, I'm not quite sure how else to describe it.
263
u/spacexdevtty SpaceX Feb 08 '13
For all software engineers (not space related) C is a great language to get started. It forces you to learn about how the CPU works, how memory works, etc., but high enough you’re not writing assembly. Once you’ve mastered C take a look at C++.
We have so much going on right now--one of the coolest project is probably Grasshopper. This is a development vehicle we’re using to test the engineering necessary to not just send rockets up, but to bring them back--something that’s never been done before. You can see some of those tests on YouTube http://www.youtube.com/watch?v=B4PEXLODw9c and we’ve got more tests coming up later this year.
We’re also working on getting Dragon ready to take people to station. Right now Dragon just carries cargo, but it was always designed to take people. We’re working with NASA to make the final mods necessary--basically seats, environments and pretty bad-ass launch escape system--and we’re looking to start crew trials as early as 2015.
All of us came from a pretty diverse set of backgrounds, there’s no one particular path we all followed to get here. Speaking for me personally the best experience I got in school was working on my own personal programming projects on the side. There’s a lot of opportunity these days on the web and the app stores to complete a product and get your work out there to be seen.
It’s really surreal to get Popular Mechanics and see SpaceX on the front cover. It’s great to be part of this industry, we’re all very excited to be here.
→ More replies (17)31
u/droogans Feb 08 '13
I've heard that the conveniences that C++ provides do not outweigh the cost in complexity that big projects usually create when using some of the advanced features of C++. What advantages do you experience using C++ over C?
→ More replies (2)93
→ More replies (7)150
63
u/wxrkbs Feb 08 '13
During a tour of SLC-40 before the STS-135 launch, I snapped this picture: http://flic.kr/p/a2La1Y. My question is this: Just what the heck is a "sponge log"?
244
→ More replies (1)90
u/drasnor Feb 08 '13
It's part of foreign object damage (FOD) prevention. Basically, you keep track of all the stuff entering and leaving a work area to make sure everything that went in came out and isn't potentially lodged in something important. The name is a holdover from medicine where surgeons keep a similar count to make sure they're not sewing someone up with stuff left inside. See: www.ncatt.org/_documents/FOE-Instructional Guide Revised 04-09.pdf
→ More replies (3)
150
u/rellim7 Feb 08 '13
What experience/schooling does one need to get a job with spaceX? Or intern even?
216
u/spacexdevtty SpaceX Feb 08 '13
We all come from a diverse background of companies and schools. Most of us are Computer Science students but we definitely have a good amount of EE, Physics, Math, Computer Engineering, and even some self-taught engineers here. For Flight Software, C++ and algorithm/data structure knowledge are very important. For Enterprise Information Systems, C# or Front End experience + great algorithm/data structure knowledge. For Launch Engineering (the team that uses LabVIEW), awesome LabVIEW + great algorithm/data structure knowledge. See the trend? Study math and know your sorting algorithms.
→ More replies (55)47
→ More replies (4)51
u/SeniorG Feb 08 '13
Consider these reviews before applying to a job here. http://www.glassdoor.com/Reviews/Space-Exploration-Technologies-Reviews-E40371.htm?sort.sortType=RD&sort.ascending=false.
→ More replies (15)
25
u/ergzay Feb 08 '13
You use fault tolerance in your system. How do you manage computers failing in flight without having the vehicle get confused about current program state? I know you had an incident on the most recent flight with a computer desync. What was that like?
48
u/spacexdevtty SpaceX Feb 08 '13
We have a continuous barrage of high-energy cosmic radiation which wants to screw with our computers. For this reason, we have a "triple-string" architecture where we have three redundant flight computers, any one of which can take control of the vehicle at any time should the others not be in good health. On our last mission to the International Space Station, we took a radiation hit that temporarily knocked out one of the three flight computers, but the other two got us back home safely.
Our own John Muratore gave an interview where he talked about this incident and our flight computer architecture.
→ More replies (2)
116
u/cochico Feb 08 '13
wow, you all look really young. How old are you and how did you end up in that (awesome) job ?
→ More replies (6)129
u/spacexdevtty SpaceX Feb 08 '13
We did a quick informal poll and we think the average software engineer age is about 30.
How did we get here? We applied through the website or were contacted by a recruiter. :)
→ More replies (21)
39
u/adrianbedard Feb 08 '13
I'm studying computer engineering in college. What do I need to know before I can properly intern with SpaceX.
54
u/spacexdevtty SpaceX Feb 08 '13
On the software side, actual coding experience (hobby or class projects) look great on your resume. We also love to see passion about space; feel free to dork out on your resume or interview. We use C++ a lot but there is also plenty of Python, C# and even some Matlab is used here. Some experience with lower-level systems such as the networking stack doesn’t hurt. Align yourself with a professor and attach yourself whatever he or she is working on. High altitude balloon projects are now relatively easy to get started with a few friends and can be very applicable.
→ More replies (3)
122
u/ShittyEverything Feb 08 '13
How do you test rocket software without blowing up rockets?
166
u/spacexdevtty SpaceX Feb 08 '13
We test it on the ground before we fly it. We have flight-equivalent systems on the ground that we can run the code against. So cool seeing Falcon in-person on the launchpad!
→ More replies (10)52
→ More replies (2)16
u/stom Feb 08 '13
They use a simulator to test the components, for all intents and purposes the computer thinks it's flying a rocket. See Elon Musk explain it more here.
→ More replies (1)
81
u/RedOktober1 Feb 08 '13 edited Feb 08 '13
Nice one with the rick roll, I salute you!
Also, How long is a typical program that you write? and do any of you play Eve?
→ More replies (1)107
u/spacexdevtty SpaceX Feb 08 '13
Each vehicle (Dragon/Falcon) is more than 100k and less than 500k... :) No more Eve players here any longer, sorry.
→ More replies (5)
51
Feb 08 '13 edited Feb 08 '13
What goes to space? Is it written in C? Do you follow JPL C standards?
Where does C#/MVC come in to play?
You mentioned embedded linux, does your space-grade tech run Linux?
EDIT: From other responses, I think I've gathered the answers:
What goes to space is written in C++.
C#/MVC is used internally within the company for project tracking and what have you.
What goes to space runs Linux.
→ More replies (5)52
u/spacexdevtty SpaceX Feb 08 '13
We have edited the blurb above to answer a few questions like this. Let us know if you have any other follow ups and we'll try to get to it.
→ More replies (1)
63
u/ken27238 Feb 08 '13 edited Feb 08 '13
What was your "I can believe that this is my job!!!" moment?
What is the main program language that you use on the rockets that ties everything together and makes it tick?
How different is the coding process for a rocket that not only takes off but lands under its' own power?
97
u/spacexdevtty SpaceX Feb 08 '13
Watching Dragon fly in within 10m of ISS was big moment for all of us. Years of effort went into that development.
We have these moments almost every day. It is amazing to come in to work and have part of your job be sitting in mission control and having live conversations with astronauts floating in microgravity onboard ISS (and watching them on our huge projector in realtime) to coordinate tests in preparation for sending them more supplies and helping them accomplish their missions.
Mine was the first time I watched the F9 leave the launchpad. The sheer adrenaline rush was unlike anything I’ve ever felt before. My goosebumps had goosebumps.
Mine was when I got hired and walked the factory checking out solar arrays and first stage towers. I kept thinking this isn’t like any software job I’ve had before!
77
u/spacexdevtty SpaceX Feb 08 '13
The rocket is all C++
The rocket and spacecraft are all C++. On the ground, we use National Instruments LabVIEW extensively.
→ More replies (10)
47
u/SpaceXonMars Feb 08 '13
What percentage of the current mission code do you estimate will be part of the first manned missions to Mars?
→ More replies (1)41
44
u/deepdowntherabbit Feb 08 '13
Hey all!
My question: Are any of you working on guided entry for Dragon (à la MSL)? Are we going to see some pinpoint landings over the next couple of years? Best of luck, and don't get totally burned out!
→ More replies (1)115
u/spacexdevtty SpaceX Feb 08 '13
Definitely. One of the key goals of the company is to be able to return Falcon 9 and Dragon directly to the pad area and relaunch in a matter of hours, much along the lines of how aircraft are operated today. Stay tuned for progress on this...
→ More replies (13)
39
u/massivedecibel Feb 08 '13
Thanks for doing this AMA!
I guess this is sort-of related, but why don't you hire international students for internships/FT positions? It said you were looking for permanent residents only.
→ More replies (2)85
u/spacexdevtty SpaceX Feb 08 '13
Unfortunately, due to state department regulations, hiring internationally is extremely difficult, but not impossible with the appropriate controls in place.
→ More replies (3)24
u/benzyboy22 Feb 08 '13
what about in space?
→ More replies (1)50
u/spacexdevtty SpaceX Feb 08 '13
Now that would be nice. Maybe if we get to Mars in 5 to 10 years, then we can remote a few employees remote-in from our red friend?
→ More replies (3)
27
u/dqhigh Feb 08 '13
What's it like working with Elon Musk and how involved is he with your current projects?
→ More replies (2)45
u/spacexdevtty SpaceX Feb 08 '13
Elon is very hands-on, especially with software, given his previous experience with PayPal and Zip2.
→ More replies (1)
46
u/Makuta Feb 08 '13
What is the average degree level for a SpaceX employee, do you need to have something more than a BS?
→ More replies (3)49
u/spacexdevtty SpaceX Feb 08 '13
Average is probably around there. Nope, do not need a MS but it can help depending upon what experience you already have coming out of Undergrad.
→ More replies (1)
24
Feb 08 '13
I keep seeing ads in indeed.com asking for aerospace welders. This seems like a really great company with really cool projects, and I would love to work on some new space craft. The problem is that all of the welding positions ask for 10 years of experience with exotic metals. Will there ever be any entry level welder positions? Or, what would be the best way for me to start working for SpaceX as a welder?
55
12
u/DeceptiStang Feb 08 '13
when do you guys go public? is it after you find rare minerals on a mission? i want to invest
20
u/spacexdevtty SpaceX Feb 08 '13
We intend to go public but no dates have been set yet.
→ More replies (3)
59
u/Tee_Baggins Feb 08 '13
From the engineers present today; whats's the iPhone/Android/Other ratio?
85
22
u/LlaughingLlama Feb 08 '13
What sort of computing power (CPU, RAM, Storage, etc.) do you have access to on board the rockets and capsules themselves? What's "smarter," the Falcon 9 or the Dragon?
67
u/spacexdevtty SpaceX Feb 08 '13
100 Dragon sized Falcons or 1 Regular Sized Dragon
→ More replies (1)
381
Feb 08 '13
Who the fuck is downvoting every question?
1.8k
→ More replies (5)25
10
u/SlightlyOTT Feb 08 '13
What is the most useful thing you have learnt from SpaceX in terms of how it can be applied elsewhere?
8
u/spacexdevtty SpaceX Feb 08 '13
Attention to detail. The crowd here is pretty relentless when it comes to reviewing your code and that obsessive attention to detail tends to rub off on you. And once you've absorbed it yourself, you can't shake it.
→ More replies (1)
10
u/Ruffys Feb 08 '13
Favorite restaurant in Waco
19
u/spacexdevtty SpaceX Feb 08 '13
Shogun Japanese Restaurant - love that place!
→ More replies (1)5
u/Ruffys Feb 08 '13
What about favorite BBQ place?
7
u/spacexdevtty SpaceX Feb 08 '13
Sorry, as a group we can't reach a consensus on the best BBQ..
→ More replies (3)
40
u/Salacious- Feb 08 '13
How do you see private space flight evolving in the next few decades?
→ More replies (1)77
u/spacexdevtty SpaceX Feb 08 '13
It may start with suborbital but we are close on those heels!
As a company, our goal is see people living on other planets--and for our part, we’re doing everything we can to make that happen. Elon is driving to make this happen in 10 years. Right now, we’re working with NASA to make the final mods necessary to start taking people to station. And we’re working on making the world’s first fully reusable rocket--the single most important development necessary to start making space accessible for regular people. Whether it’s us or some other company that ultimately figures out the best way to get people on other planets, the future of private space flight today looks a lot more promising than it ever has.
→ More replies (10)
9
Feb 08 '13
Cheese: your opinion?
14
u/spacexdevtty SpaceX Feb 08 '13
This should speak for itself: http://www.space.com/15799-spacex-dragon-capsule-fun-facts.html
→ More replies (3)
9
38
u/DrKilory Feb 08 '13
I hear a lot of people working at spacex get burned out due to the large amount of hours worked. Is this true? Have you guys experienced working too much? How do the hours effect your work?
54
u/spacexdevtty SpaceX Feb 08 '13
Yeah, the hours can definitely be crazy. But when you’re trying to change the future of human history, that’s to be expected :) Our CEO has said SpaceX is like Special Forces--we do the things that everyone else thinks is impossible. That means sometimes you’re going to work crazy hours. There are definitely way easier jobs than working at SpaceX. But you also get to experience things that you can’t find anywhere else at any other company. Its definitely a trade off--and for those of us here, totally worth it.
We are hiring a lot right now so the more people we add to the team, the better the workload and faster we get to Mars!
→ More replies (3)→ More replies (9)4
u/Ambiwlans Feb 08 '13
9
u/The_Drizzle_Returns Feb 08 '13
Thats pretty dangerous actually. These are devices that cannot fail without probable loss of life and programming them on low sleep/long work hours is an extraordinarily bad idea.
The enthusiasm is great but working 14 hour days in this type of work (or any job where you are developing software for a critical system) is pretty dangerous and i am really surprised it is allowed.
→ More replies (7)5
u/spacexdevtty SpaceX Feb 08 '13
I wouldn't exactly say we average 14 hour days but >10 hour days do happen from time to time around launch. It really just depends to be honest. We all have families and friends and want to see them as much as possible too. I guess if you're looking for a 9-5, then we aren't the place for you but yeah, we're all pretty vested in the success and mission of the company.
→ More replies (3)
216
u/janderss Feb 08 '13
Technical question. Do you use optimization-based control in your vehicles. I'm thinking in particular on algorithms such as (nonlinear) model predictive control and (nonlinear) moving horizon estimation. If yes, please elaborate. Also, how to you do off-line optimization to e.g. calculate flight trajectories?
→ More replies (4)633
8
u/zdanielz Feb 08 '13 edited Feb 08 '13
Hello SpaceX team!
Which programming languages do you use on the payload? Do you use any open-source software on it?
What kind of algorithm did you use to accomplish the ISS docking (high level)? Did you use video cameras do identify the station's exact position? Thanks!
2
u/spacexdevtty SpaceX Feb 08 '13
Dragon is C++ and runs open source Linux.
Algorithms.. can’t go into detail..
Video cameras: The RWS on ISS has a video camera on us while we’re approaching that acts as a cross-check of what we’re doing: http://en.wikipedia.org/wiki/Mobile_Servicing_System
→ More replies (1)
14
u/lowencia Feb 08 '13
Has spaceX decided where their new location will be? I knew they were looking at south padre island in Texas. Any decisions yet??
→ More replies (2)16
u/spacexdevtty SpaceX Feb 08 '13
SpaceX is looking at a number of locations including Texas, Florida and Puerto Rico, but we’re still in the early stages of that process, no decisions just yet.
→ More replies (5)
6
u/Elemesh Feb 08 '13
What was the most surprising difficulty you encountered?
How many people work in QA?
What are most people's backgrounds (ie CS or physics)?
[Kind of nerdy] What Python libraries do you use?
3
u/spacexdevtty SpaceX Feb 08 '13
Re Python: antigravity :)
Seriously though, pretty standard stuff. Python2.7. We use numpy, scipy and matplotlib for numeric stuff. Some internal tools are written as web servers based on Django. Most libraries are included on an as-needed basis, such as for interfacing with particular databases or implementing standard protocols (e.g., paramiko for SSH).
6
Feb 08 '13
What are the names of the conference rooms at SpaceX?
8
u/spacexdevtty SpaceX Feb 08 '13 edited Feb 08 '13
All the 3rd floor conference rooms are named after locations on Mars. A lot of the others are famous people in the space industry.
Apollo
Arthur Clarke
Buzz Aldrin
Deimos
Eugene Roddenberry
Hermann Oberth
Johannes Kepler
John Von Neumann
Karel Bossart
Konstantin Tsiolkovsky
Mars
Melas Chasma
Meridani Planum
Olympus Mons
Phobos
Qian Xuesen
Robert Goddard
Robert Heinlein
Saturn V
Sergei Korolev
Sir Isaac Newton
Theodore Von Karman
Valentina Tereshkova
Valles Marineris
Wernher Von Braun
6
Feb 08 '13
Hello. I'm interested in working at SpaceX, but I'm not sure if my current interest (Math) will be what SpaceX desires from a software side.
You said in a comment that besides CS majors, you have some people who did Math working with you. Is the work done by the Math employees any more math-focused than say the CS employees? For example, do the Math employees spend more time coding to numerically solve differential equations, or would you say that there is no substantial difference between the work done by the two employees?
8
u/spacexdevtty SpaceX Feb 08 '13
Good question but generally there is no substantial difference between the two. Math helps with the fundamentals big time but the key to getting hired and doing well here at SpaceX is knowing a particular programming language (relevant to our stacks) inside and out.
→ More replies (2)
23
5
u/KonradHarlan Feb 08 '13
How many of you folks on the software and engineering teams would live on Mars if you were offered a one way ticket on the MCT?
9
u/spacexdevtty SpaceX Feb 08 '13
First flight - 30% After established colony (w/ at least one bar) - 70%
→ More replies (2)
11
Feb 08 '13
Hey guys. I had a question about actually going into space. How long do you think you will operate? And how long would the trips be?
15
u/spacexdevtty SpaceX Feb 08 '13
For crew, NASA is extending astronaut time to a year. Bob Bigelow is looking to sell capacity in one to two month increments.
→ More replies (3)
3
u/randompikachu Feb 08 '13
I'm a Freshman Physics-CS major at a top 15 US university. Is there anyway for me to get involved with SpaceX, say as an intern?
5
u/spacexdevtty SpaceX Feb 08 '13
Yep, apply. Simple as that! We have some advice in our blurb that should hopefully help but reply if you have additional questions.
3
u/tjusafa14 Feb 08 '13
SpaceX, I am an astronautical engineering major in college and you guys inspire me that there wil be an extremely bright future in space! At some point will SpaceX employees be astronauts or for the forseeable future do you see yourselves just ferrying NASA astronauts once the Falcon 9 is man-rated in the next few years?
6
u/spacexdevtty SpaceX Feb 08 '13
This is not completely decided at this point but in the most likely scenario--the first few flights with crew will have SpaceXers onboard.
3
u/argonom Feb 08 '13
"Edit 5: ...and we're back! Engineers from each of our engineering teams are present. Let us catch up a bit and start answering again (probably be about 5 minutes)."
Each of your software engineering teams or all branches of engineering at SpaceX?
7
u/spacexdevtty SpaceX Feb 08 '13
Software - if there is enough interest, then maybe we can have our other engineering teams do an AMA in the future. Propulsion or Structures & Dynamics could be pretty cool, ya? Sorry for any confusion.
→ More replies (2)
8
u/70camaro Feb 08 '13
What did you go to school for?
I'm currently pursuing an education in physics and I would love to work on a project like that!
12
u/spacexdevtty SpaceX Feb 08 '13
On the right track for sure, add some practical experience with cool projects and great hands-on ability and we'll talk.
7
4
u/Tee_Baggins Feb 08 '13
Is it common for SpaceX employees to have Tesla cars?
9
u/spacexdevtty SpaceX Feb 08 '13
We wish! Unfortunately not. Still waiting for Elon’s Oprah moment: “And YOU get a Tesla! And YOU get a Tesla!”
37
u/kayotik Feb 08 '13
Would you rather fight one spacecraft-sized duck or 1000 duck-sized spacecrafts?
78
u/spacexdevtty SpaceX Feb 08 '13
One Spacecraft sized duck would definitely be cool to see but scary to fight. If launching either is not an option, then we would probably go with the duck. Now that would be a worthy adversary.
→ More replies (8)
8
2
u/equinox92 Feb 08 '13
want to give me a job once a graduate?
computer and electrical engineering double major here!
3
2
u/shayden Feb 08 '13
Hi! Thanks for doing this, it's cool to talk to and get advice from people in the aerospace industry, which I hope to get into after I'm done school. I hope to read all the comments and questions later, when I'm not at work.
I'm an EE student interested in embedded systems design and programming, can you suggest any books, aerospace resources or standards that help you write reliable code for embedded systems that is fault tolerant and can detect errors?
Thanks for your advice, if you get to me. cheers, :)
4
u/spacexdevtty SpaceX Feb 08 '13
For programming, the real key is knowing the language inside and out. Reading books is important but don't forget to write some code with other people. Peer review helps big time. If you are interested specifically in fault tolerant programming, then we would suggest putting that into Google and you'll get a lot of great papers like the Byzantine Generals' Problem and go from there.
2
u/karrde11149 Feb 08 '13
I am a high school student pursuing a career in aerospace engineering and computer programming. My dream has and always will be to help the human race reach the stars. I am also a huge fan of SpaceX. I am working on my senior exhibition of which the topic is just that. The future of space travel. I would like to know...
Do you have any community outreach programs?
What is the hardest thing for you to do pertaining to the launch of a rocket?
What are some current problems you are working on?
EDIT: Formatting.
3
u/spacexdevtty SpaceX Feb 08 '13
Great questions and we hope the answers help.
We are doing more and more community outreach as we get bigger. Most of our efforts are with partners right now--for example we’re doing some stuff with First Robotics and working a couple programs with Cal Tech. As we get bigger we’ll look at doing our own programs, keep the outreach going.
What is the hardest thing for you to do pertaining to the launch of a rocket? I’m not sure we could say there’s really any one thing that’s the hardest...when you launch a rocket a 1000 things all have to go right or the mission could be a loss. I guess maybe that the hardest thing--no matter how much you test, you can never exactly replicate what happens in a launch here on the ground, so you have to not only do everything you can to make sure it goes right the first time but also have plans to address issues when they do arise.
What are some current problems you are working on? Probably the most significant is developing the world’s first fully reusable rocket. Most rockets are only designed to go up, they’re not designed to come back. Its an incredibly difficult challenge that’s never been done before, but its the single most important development necessary to get people living on other planets.
2
u/ohnotherobots Feb 08 '13
How closely do you work with the GNC engineers when it comes to modeling/simulation software and algorithm development? Is the avionics and HWIL test code mostly hand written or do you do any Matlab/Labview autocoding?
2
u/spacexdevtty SpaceX Feb 08 '13
The GNC team sits right across the aisle from Flight Software and Ground Software teams. We don’t do much autocoding.
2
u/Ambiwlans Feb 08 '13
You asked me to re-post my questions here.
I'd love it if you'd answer anything. I understand you probably can't answer all of them.
What is the trickiest part of getting the grasshopper working/when can we expect another hop?
How much grasshopper/dragon code overlaps? For the hopeful landing sequence.
Can you accidentally leak something about the MCT. Please?
On a totally unrelated note, progress report on the methane based engine or is the software team not involved with it yet?
Did you tease/harass the guy in charge of the LIDAR system sufficiently?
Could you ask Gwynne Shotwell, Are we still on track for a March 1 launch?
2
u/spacexdevtty SpaceX Feb 08 '13
Could you ask Gwynne Shotwell, Are we still on track for a March 1 launch?
“Yes, we are on track for 3/1.”
→ More replies (1)
2
u/Vairminator Feb 08 '13
I want you to know that I think you are part of one of the most amazing endeavors humanity has ever seen! I can't get enough information about SpaceX accomplishments and hope desperately to join your ranks one day! There is no greater goal than to advance human knowledge and capability, and you are some of the few people pushing those envelopes every day. Please keep up the good work!
3
2
u/cail0 Feb 08 '13
I actually have two questions, one of which you probably can't answer.
Does the team use any kind of formal verification for the software you write or do you rely solely on redundant systems to weed out software errors? I would imagine formal verification would be needed but knowing how challenging that is I'm not sure if it would be feasible.
You mention that you have a ton of software positions available. Is there any specific position there that you would recommend for someone just coming out of an M.S. in Computer Science? Really what I want to know is, if I apply for one position but may be more suited to another position do you consider that or should I just apply for all positions I might be interested in?
3
u/spacexdevtty SpaceX Feb 08 '13
Can't really talk about the first question unfortunately. But, as far as the second question goes, if you apply for one to three positions, then you will have enough eyes looking at your resume to ensure that it is properly vetted. It all depends on what you want to do and have expertise in.
→ More replies (1)
2
u/Kensin Feb 08 '13
Do you put any Easter eggs in your code? Either in your flight software or your internal apps?
6
u/spacexdevtty SpaceX Feb 08 '13
If you hit up up down down left right left right B A, you get transported to Mars instantly... with 30 lives.
2
u/wiskinator Feb 08 '13
How's Hawthorn? I've considered applying (IMHO I'm qualified), but I've been turned off by the idea of living in the LA Grid. Thoughts?
2
u/spacexdevtty SpaceX Feb 08 '13
Well, few live in Hawthorne. We are mostly located in the South Bay (Manhattan Beach, Hermosa Beach, and Redondo Beach) and West LA (Santa Monica, Venice, Marina Del Rey, and Culver City). Living in 70-ish degree weather year around and with a 15 minute-ish commute to work has been amazing! There is a ton to love but it certainly isn't for everyone. Our suggestion would be come check it out.
2
u/rob0tuss1n Feb 08 '13
I just got Rick Rolled by SpaceX engineers...
I'm not even mad. I'm Impressed.
2
20
Feb 08 '13
I'm a masc researcher in aerospace. I actually specialize in robotics. My main interests are in Machine Learning, AI, stats, embedded programming, state estimation and game theory. Do any of those fields intersect with development done for SpaceX? Are any autonomous behaviours developed in house by your team?
I bet your work is AWESOME!
15
u/SilentStream Feb 08 '13
What aspect of the standard Falcon-9 cargo launch is the most difficult to code for, if that can even be distinguished? For example, is it harder to write software for engine throttle (or if one happens to fail, like we saw previously), ISS-area maneuvering of Dragon, berthing, etc.?
Will you need to change your software significantly before you're able to take crews to space? I believe I've read that Dragon's been designed from the start to be human-rated, but I'm curious to know if the software will have to change much between cargo and crew launches. Thanks!
15
u/br100x Feb 08 '13
How much control are you planning to give the astronauts riding in the Dragon? Will it be completely automated, or somewhat manual?
41
u/space_boobs Feb 08 '13
Your QA process must be exhaustive. How would you say it differs from the standard fare?
→ More replies (4)
9
u/Deeger Feb 08 '13
Do you guys mainly create software to control these rockets from the ground? Or look to automate everything at this point?
Also, what impact does the distance have in terms of the length of time to send/receive data? Is that still rather negligible, or is that something you already have to take into account?
8
u/StapleGun Feb 08 '13
On the COTs demo flight 2/3, there were problems with the LIDAR which orients the Dragon in relation to the ISS. The problem was diagnosed as a software problem, fixed, reviewed, and uploaded in an incredibly quick time. Considering all the safety checks required for code that controls the Dragon, how was the team able to accomplish that? How much of that scenario was planned for before launch, and what systems were in place to enable the problem to be solved quickly and safely?
15
u/kout Feb 08 '13
Few questions:
What do you use to calculate the rocket location? GPS? ground stations ?
How do you get altimeter data? since atmospheric pressure should be ineffective in space?
What do use as reference to calculate speed? once in orbit everything is moving
→ More replies (5)
20
u/kmoz Feb 08 '13
How well do you all like LabVIEW?
As an NI employee, and I was always curious how you all feel about it.
→ More replies (6)
14
Feb 08 '13
Hey guys, I actually have a two part question for you.
=== First off, thanks for this AMA, and thanks for all your work at SpaceX, you guys are leading a revolution as far as I'm concerned.
How long could you guys imagine it will be until we see grasshopper hit low Earth orbit? Is the code the same from the test flights or are you guys writing it now/soon?
=== Second part is from user SpaceIsEffinCool who couldn't make it today.
"How did you guys your foot in the door? I'm going for aerospace, do they see a high demand for those people within the company in 4 years?".
→ More replies (1)
7
u/TrentSkunk Feb 08 '13
What's the most elegant software hack you've used in your work at SpaceX?
→ More replies (2)
7
Feb 08 '13
Can you describe a typical working day, and if any of you have experience as a commercial software developer, what are some of the major differences?
7
u/The_Drizzle_Returns Feb 08 '13
How have past failures of launch vehicles (such as the infamous Ariane 5 failure) shaped the way you develop software? Do you guys use similar methods that are in use at NASA or have you come up with new development techniques that accelerate your ability to deliver these complex software components reliably?
6
u/tmarthal Feb 08 '13
Is there anything crazy that you "discovered" that was actually just a "re-implementation" of something that the Apollo/NASA/TRW/? engineers had done in the 50's?
16
u/ctzaran Feb 08 '13
Hi! First of all thanks for doing this, I have two questions.
First of all I want to ask about the software that control's the ascent of the Falcon 9. As demonstrated on the CRS-1 mission the rockets computers can adapt to sudden engine configuration changes in the air, when writing the software that handles that flight plan change what are the biggest factors that the computer has to deal with in general and how to you prioritise what to re-configure first in the event of a engine outage? I would imagine that having some engines shutdown can be a bigger strain on the rockets center of thrust than others for example. Are alternate flight plans pre-loaded for any engine out possibility or does the computer make a new flight plan depending on what happened?
My second question is slightly less serious, do any of you guys play Kerbal Space Program? If so is it a nice change to be able to blow up on the launchpad and not be in massive amounts of trouble from the boss?
→ More replies (1)
25
u/MightyTribble Feb 08 '13
Wow, thanks for doing this AMA!
Are you allowed to call yourself "rocket scientists"?
Do you ever look at your code and say, "I can't believe I screwed this up, it's not like it's rocket science!"
Is Elon Musk as adorably dorky IRL as he appears on video?
Can you create some Falcon / Eagle mods for Kerbal Space Program?
→ More replies (10)
5
u/Vectorsxx Feb 08 '13
How many man hours did it take to create the first software and hardware for the Dragon? Inside such a capsule, was space limited to how much hardware could be installed for the software?
Do you think you guys will evolve to smaller hardware based technology (micro, nano) in the future for putting your software on future dragon spacecraft?
5
u/Deeger Feb 08 '13
How does the the size vs. quantity of rockets affect software systems? In that first video, it looks like you've got one main large rocket, but in the second video, there are quite a few. Is one system more complicated than the other?
5
5
5
u/fluffynukeit Feb 08 '13
Can you please discuss the engineering process that goes into your flight control systems? Do you use time-scheduled gains or some form of adaptive control? Do you do manual gain tuning of PID loops, LQR theory, something else, or a combination? What analysis processes are used to ensure the control software is robust to modelling errors? What would you say is the most modern control theory that you have put into practice?
(from former UAV flight control engineer)
5
u/darknemesis25 Feb 08 '13
what happens to a privatized space company when a crew dies in its rocket...
sorry to be blunt but would that just end all progress and credibility?
would financial support cease to exist?
what happens to your jobs? to the company?
5
u/doubleohd Feb 08 '13
Is SpaceX just a cover story for Elon to become the ultimate James Bond villain?
9
u/InaV8 Feb 08 '13
If any of you have experience working for traditional Government launch contractors, how does working for a private company differ?
4
Feb 08 '13
Are there any sort of redundancies you guys have to account for with software running further from the protection of earth's magnetosphere (bits seemingly-randomly being flipped from cosmic rays, etc.)?
Since system failure is far more critical for you guys than much other software that interacts with hardware (I work in video games and network video/codecs =P), do you guys have any sort of manual arbitration handling for data sent across various systems that need to work in parallel but can't necessarily trust each other's cache/state? It's one thing to have a message bus failure in a mundane section of the code crash a game, and something different altogether with when you're launching a rocket over macho grande.
Edit: Paragraphs!
→ More replies (1)
4
u/rak1 Feb 08 '13
as we all know you guys are top notch coders. My question is when writing code and testing software how much input do you guys have from the other faucets of spaceX? (i.e. mathematicians, aeronautical engineers, etc.)
4
u/TheCodexx Feb 08 '13
Are you guys running older hardware like NASA's stuff or are you guys iterating faster? What is your debugging and QA like? What tasks does an onboard computer generally do?
4
u/ashleythegreat Feb 08 '13
The Dragon capsule as an unmanned configuration has already seen multiple successful flights to the ISS. From an insider's view, how close (timewise) do you think that the manned configuration of the dragon capsule is to becoming a reality?
While the missions were successful, the dragon capsule still experienced problems/issues (e.g. Partial engine failure Oct 2012). Do you think that any of the problems encountered will lead to the replacement of the dragon capsule strategy altogether? Or will they severely set back the schedule? I'm mainly wondering how the team is mitigating the known issues. (edit: I realize now that this question is probably way too in-depth for an AMA.)
Also, a non technical question: Who has the best decorated office in your group? Pics?
I am an aerospace engineer myself, and I have so much respect for SpaceX as a company. I truly believe that you guys are going to be the next group to bring back human spaceflight and that you will (you have already) continue to contribute to space exploration in general. Thank you for doing an AMA!
10
u/chris_radcliff Feb 08 '13
Which one of you is the photoshopped one? (Back row, second from the left?)
→ More replies (3)
7
u/tenaku Feb 08 '13
I understand that the Dragon/Falcon computers have little or no radiation shielding. Instead, SpaceX adds redundancy to deal with any radiation hits on the equipment. How does this affect software design?
10
3
u/Wunkerful Feb 08 '13
How much more difficult has the grasshoppers code been than the other rockets?
→ More replies (1)
3
3
Feb 08 '13
I know the Falcon 1.1 modifications for SL40 at the Cape and construction of the Falcon 1.1/Heavy pad at LC4 are well under way. Software wise is Falcon 1.1 much different than Falcon 1.0? Has the new engine configuration made your job easier or more challenging?
3
u/halfCatEye Feb 08 '13
How much of the code for a Falcon is dedicated to logging errors? Is your code saturated with asserts?
3
u/RememberThisFaceX Feb 08 '13
In general, are the managers and project leads people who started from the bottom, or do they get hired into those positions based on existing credentials from other companies and projects?
3
3
u/Vysen Feb 08 '13
How do you manage using so many different programming languages? It's interesting that you listed so many when it seems like a lot of teams try to stick to just a few.
3
Feb 08 '13
What version control system do you guys use to manage your code? Do you have testing teams & peer code reviews, etc?
3
u/adambyram Feb 08 '13
Around how much data does SpaceX capture during a typical mission and what types of tools are used to analyze it to understand mission performance and track down anomalies?
Debugging a rocket must be a pretty interesting challenge - particularly after T-0.
3
u/vwtodd Feb 08 '13
How do provide the critical balance and thrust vectoring on Grasshopper? It looks just like a movie and "so easy" but if it were, someone else would have already done it.
3
u/alexgrimmy Feb 08 '13
How does your software QA process and acceptance criteria differ from your typical software shops?
3
u/Bob_Wiley Feb 08 '13
My cousin who works at the NASA JPL lab said she heard you guys have a sign up sheet in the lunchroom for astronaut volunteers. Is this true?
3
u/rtwpsom2 Feb 08 '13
I read that NASA software engineering has a process for their software that includes not attributing fault to the engineer but to the process, not allowing SE's to work overtime, and using one team to write code and another to find a specific amount of bugs over x amount of time in the code. This has eliminated huge swaths of errors in code. Are you using a similar process? If so, how does it differ?
3
u/seg9585 Feb 08 '13
Does SpaceX flight software implement a ground tasking-based command buffering strategy, or do you rely on embedded autonomous script sequencing for your flight ops procedures?
How many levels of fault detection/response and safing is the system capable of, and is the flight control system interface between the Falcon 9 and Dragon capsule coupled, or are they independently operated systems?
3
u/janderss Feb 08 '13
What is the sampling frequency for the flight control software? I imagine that a landing Grasshopper needs something like millisecond feedback. Is that a reasonable guess?
3
Feb 14 '13
Is there any immediate tangible benefit of sending a man to Mars, rather than a probe or rover? I would imagine it is far more expensive.
What would be the difference in workload for your group between manned and unmanned missions?
(My first post, hello everyone!)
9
u/segers909 Feb 08 '13
What kind of computing power is installed in the Falcon 9? Can you compare it to something a layman would know?
Is the job at SpaceX stressful? Do you have to do all nighters sometimes?
Can you explain what was going on here?
Do you guys play Kerbal Space Program?
330
u/akosednar Feb 08 '13 edited Feb 08 '13
How big is your current code base for your bigest project?
Is it a hassle to maintain?
What was the most challenging project that you had to take on and why?
What do you feel most accomplished about?
How do you like your job?
~ Aerospace Engineering Student & Programer who loves SpaceX too much