I'l be live tweeting @B777Alex's Bauduin's keynote here at #EuroSTARConf. Stand by.

"Flying High: Test Automation in the Boeing 777" was preceded by conversations between Air Traffic Control at JFK and pilots.

As a 777 captain and simulator tester, Alex is talking from experience.

A flight simulator requires realistic simulations, which requires creating realistic environments, including air traffic control, weather, other pilots and planes, failures, GPS, physics, aircraft systems; everything deemed relevant for training.

Pilot training starts with a laptop, and works its way up to a simulator that has a 20-year lifespan including equipment, networking, software development including the toolchains in order to replicate the same system as the one delivered. It must be certified by authorities.

The new regulations starts at Page 18117 in the documentation, which seems a little daunting.

Desktop flight simulators are used to train not only pilots, but also maintenance people. Next is a procedure trainer; lots of checklists that must be learned and run efficiently.

Procedure trainers involve illustrations of particular sets of switches and controls on a screen.

(Alex is here in a captain's uniform to represent two roles: user and engineer. The goal is to emphasize the significance of deep user knowledge.)

Here are two pictures, one representing the cockpit and windscreen view of a real plane; the other a 3.5 ton simulator. They're indistinguishable. The sim, again, is used for both pilots and maintenance people. It must represent the whole feel of the real plane.

Sim builders need to work with lots of specialists: electronics, avionics, electronic motors, wiremen... all this in order to convey the feeling of a real plane in a real environment. This includes simulating the ground: tar, pebbles, thin layers of ice and water.

Tiny variations in the amount of water will change the grip of the tires. Tiny amounts of ice on the engine will affect engine performance. Landing gear stress has to be modeled to get the exact same feeling as the real thing. Alex took the plane to FL430 (43,000 feet)...

...on the sim, where he experienced fuel pump cavitation; the behaviour of liquids in pressure changes.

The sim must model the landscape from 15-50km from the airport. Even signs on hotel buildings must be represented so that pilots can recognize landmarks.

The maintenance computer on the sim sees a real plane, with valves, relays, sensors, computers. Run the test on the sim; the maintenance computer won't know the difference between the sim and the real plane. The electronic bay on a 777 is 4.5 metres long. There are LOTS of boxes.

Computers for braking, engine controls, all the stuff on the plane.

What is NOT simulated? Ocean tides; ACTUAL pressure changes; passengers screaming (!). Sim builders build and test 20+ million lines of code; 1-2 million parts, OEM and home-made; several kilometres of wire.

Complicated and complex: between systems due to the laws of physics: drag, runway drag and slope, engine thrust, rolling friction... Air conditioning, air from the engine, thrust, fuel burn, lift rate of climb are all inter-related. Crazy complicated diagrams illustrating this.

This stuff must be TESTED, not just checked. The determination of whether it's representative or not must be made by pilots who have flown in all of the operational tasks listed in this huge-ass Appendix to the regs. The painful part is the subjective tests.

It's hard to evaluate that the effects (e.g. of a blown tire) are representative of real-world conditions. There are thousands of procedures performed interactively inside the sim. The company was making 50+ simulators per year. Alex had to sign every test he performed.

Nobody could FORCE Alex to sign those docs, because HE would go to jail if something was misrepresented.

A new line of simulators, the XR line, prompted by new Arduino-based hardware. Alex took this opportunity to contribute testability to the design of the sim.

In this process, he discovered the principles of the context-driven school of software testing. A favorite, for him, was item 7. (YOU look it up. --MB) Alex's expression of this: Test the right thing, at the right place, at the right time (with the right tool).

Developing a new sim required crazy-strong version control, including buying 2000 versions of the same processor for replication. Lots of testing on the desktop, distributed testing, testing at the assembly level.

Objective? Subjective tests might be more objective!

Test requirements from the authorities rain to 643 pages, with crazy-detailed tolerances. Dynamic testing was informed by data injection of records from early test flights of physical aircraft. LOTS of knowledge of physics needed for explaining discrepancies between sim and real.

Alex used Fitnesse for checklists for manufacturing to help validate. This gave him an edge, knowing that this testing had been done. Human testers perform physical actions; Fitnesse helps to track the results and drive the Arduino to send messages around. Fitnesse spies on that.

Alex was thereby able to move a lot of the testing that he previously did to manufacturing, so that testing would be done by the time it got to him. Using Cynefin to help sort out complex and complicated, Alex laid out elements and factors of the test setup. (Blown tire example.)

He used the procedure for engine start and was able to get it instantiated in code. Now he can demonstrate to the aviation authority that the sim is able to represent the flight profiles. The system includes 500-900 failure simulations, many elements of which can be checked.

Using Fitnesse, a single button can get the system into a state where it's ready to simulate a particular condition.

Temperature changes with pressure and altitude. This means that some parts of the runway might be covered in water, others in ice.

You've GOT to challenge a green test, or you'll never know the truth about it.

Automation afforded opportunities to perform lots of routine stuff. But don't automate everything, or you'll end up killing yourself. -fin-