Building Drones for Developers

This summer, we wrote in Noahpinion that the American drone industry’s edge over China is its collaborative community of open-source developers — ecosystems like Ardupilot who allow the United States to leverage its advantage in software and AI. We believe deeply in the potential of innovative hackers building interesting new capabilities on drones, be it GPS-denied navigation from Palantir, computer vision for population-counting, or remote sensing to protect agricultural equipment, and applying them across American industries and on the battlefield.

That’s why we designed the F-11 as a platform for developers. In the F-11, we combine the programmability, configurability and open architecture of a research drone with the reliability of a production-grade workhorse. It’s built around a massively powerful GPU, empowering users to access a full machine learning stack in-flight. Here’s how we’re building the F-11 to facilitate software and hardware development:

#1: You Are Not Limited By The Hardware

The F-11 is designed to ensure developers are never limited by the hardware — with massively powerful onboard compute, ultra-fast solid state storage, and the battery power to support it. Developers have full access to:

• A powerful NVIDIA Jetson Orin NX GPU: 100 TOPS of processing power, enough for any application you could dream up for a robot to do.
• 500GB or 1 TB NVM of onboard solid state storage, enabling ultra-fast write speed and retrieval speed. That’s the highest developer-accessible solid state storage of any military or commercially available UAV.
• An 8-core ARM CPU with 16 GB of RAM.
• 18,000 mAh onboard Li-Ion battery capacity.
• A broad range of power voltage and port options — 5v power, 12v power, VBATT ranging 50v-38v (we know how frustrating it is when you have a 5v device you want to integrate and there’s not a 5v port on your drone!)

#2: Everything Talks to the GPU, and the GPU Talks Back

A uniquely open architecture on the F-11 means that every part of the drone is truly built around the Orin. This enables sophisticated autonomy applications in which ML models are able to not only analyze data obtained in-flight, but actually use that analysis to inform flight actions in real time — what allowed Palantir to build out an APNT capability on the platform. Additionally, the F-11 features a publisher/subscriber message bus — a shared communication hub where any service on the drone can post messages and subscribe to topics they need.

By connecting the GPU directly to the flight controller, we create a clear pathway for the GPU — and the autonomy software our developers write on it — to control all of the drone’s actions. Our true open hardware approach features USB 3.0 ports that are directly wired into the GPU, along with Ethernet ports connected to a 1 GB/s high-speed switch, ensuring seamless data transfer directly into the GPU. Our setup allows any data transmitted through remote control (RC) and all attached sensors to flow through Ethernet through the Orin, allowing the on-board compute and GPU to access data from external interface ports connected to USB 3.0 and Ethernet. This architecture ensures that the connections between nodes and data pathways are not bottlenecked, with ample onboard compute and storage resources available for demanding applications.

#3: Programmability and Adaptability in Hardware and Software

Today, commercial and military drones are built as frozen products. The end user’s ability to modify either the software or the hardware is extremely limited. Onboard compute is either nonexistent or inaccessible to developers, so if you want to run your own machine learning applications onboard, it’s difficult or impossible. The physical housing of the drone is frozen as well, with a handful of mounting hardpoints in inflexible locations — so if what you want to attach to the system is just a few inches off, you’re out of luck. Today’s drones are static.

But warfare is constantly changing — particularly in a conflict against a near-peer adversary. Tactics, countermeasures, and new advancements in technology are dynamic. It’s essential that we develop UAS capable of adding new capabilities and rapidly adapting to change, both in industry and on the battlefield.

This is the central thesis behind our flagship product. On the F-11, every aspect of the drone is designed to be as configurable and programmable to the end user as possible. All of our software services are containerized using Docker, ensuring that developers can easily deploy, update, and manage applications in a consistent and isolated environment. This containerization allows for rapid iteration, simplified dependency management, and consistent performance, no matter where the application is running.

• Flexible Power Ports: All of the power ports are XT-30s, the most easily modifiable power source in hardware design — there are no complex, bespoke connectors you need to navigate to access power on the device.

• Plug-and-Play with MAVLink: On the F-11, developers have full access to MAVLink. The system runs on the standard MAVLink protocol, without modification or custom messaging, enabling plug-and-play with new MAVLink systems.

• Smart Firmware Design: ArduPilot, an open-source software suite for UAVs and the most transparent and easily manipulatable flight control suite, is the driving force behind our flight controller. Its robustness and maturity enable developers to easily build applications that can control the F-11 at a low or high level. Unlike other flight control systems, Ardupilot is open, free-of-charge, free to modify, and transparent to the developer, with over 20,000 community members available for troubleshooting.

• Configurable Housing: Off-the-shelf, the housing of the F-11 is injection molded using high performance thermoplastic. However, upon request, we’re able to 3D-print over 100 in² of the housing with high-strength nylon to customer specifications. That means if you have new hardware addition or payload integration, you can adjust the mounting hardpoints accordingly. It also means you get to change the interface ports going into the drone to support the payload you want.

#4: The Flyby Payload SDK

On traditional UAV platforms, new sensor integration can be a lengthy and difficult process. We designed the Flyby Payload SDK to ensure that payload developers can connect to the F-11 and the controller by writing only a few lines of code. By uniquely routing signals inside the drone to permit key sensor actions, we’re able to easily enable inspection, reconnaissance, and photogrammetry missions using novel payloads. This currently supports EO, IR, gimbaled, fixed, and laser rangefinder payloads, but this list is quickly expanding to include OGI, multispectral, LiDAR and more. If you’re a developer integrating a new camera system, you shouldn’t have to know the F-11 in and out to get your sensor to work on our system. With only a couple lines of code, your sensor should be able to:

• Receive command inputs from the controller
• Share data streams to the Jetson
• Take snapshots and programmed intervals in order to conduct photogrammetry

#5: Flyby Nexus — Machine Learning Modules

Flyby Nexus simplifies the process for developers by enabling seamless updates to the Jetson Orin without the need to re-flash the system. With this application, you can effortlessly update the GPU by simply uploading a new machine learning module. Simply upload an ONNX model through Nexus, and our systems will seamlessly integrate the model into the end user experience. This allows machine learning experts not to worry about the details of video streaming or UI — just deploy a good model and our state-of-the-art hardware and software combination will instantly make it production-ready.

Flyby Nexus automatically handles incoming files and integrates them into currently active data streams, allowing for real-time operation without the hassle of managing logistics related to code execution. This ease of access makes it an ideal tool for developers who want to focus on building and deploying sophisticated ML applications without worrying about the underlying complexities of the hardware.

Interested in building an application on the F-11? Reach out to [email protected].