Early in my time working on embedded cellular gateways, I inherited a firewall implementation that was a single large shell script. It worked — in the sense that it produced a functioning firewall — but it was slow, brittle, and every time we needed to change it, something broke. Eventually we rewrote the whole thing in C++. The runtime dropped by an order of magnitude, but honestly, the performance was not even the best part.
This is the story of that rewrite and what I learned from it.
What We Started With#
The existing firewall was a single shell script. Not a small one. It would start by making calls over message queues to a custom central configuration server to retrieve the device’s current settings — what interfaces were up, what services were enabled, what the user had configured through the web UI. Then, based on all that state, it would construct and execute hundreds of individual iptables calls.
Every rule was its own iptables -A or iptables -I command. Chain creation, policy setting, NAT rules, filter rules, forwarding rules — all individual shell commands, executed one at a time, sequentially.
The performance problem was straightforward. Each iptables call is a separate process invocation. Each one has to load the existing ruleset from the kernel, add or modify a rule, and write it back. When you are doing this hundreds of times in a row, you are loading and writing the entire ruleset hundreds of times. On the embedded hardware we were targeting, this added up to a noticeable delay during boot and whenever the firewall needed to be reconfigured.
But performance was only part of the problem.
Why Shell Was the Wrong Tool#
The deeper issue was maintainability. The script had grown organically over time as features were added, and it was genuinely difficult to work with.
Shell scripting gives you string manipulation, conditionals, and the ability to call external programs. That is about it. When your logic is “if the device has a VPN tunnel active on interface tun0, and the user has enabled port forwarding for TCP port 443 to internal host 192.168.1.10, and the device is in router mode rather than bridge mode, then add these specific NAT and filter rules” — expressing that in shell becomes an exercise in nested if statements, string concatenation, and hoping you got the quoting right.
Error handling was basically nonexistent. If one iptables call in the middle of the script failed, the script kept going. You would end up with a partially configured firewall and no clear indication of what went wrong. Debugging meant adding echo statements and staring at log output.
Testing was also painful. You could not unit test any of this. The only way to verify a change was to deploy it to a device and see what happened. Given that a misconfigured firewall on a cellular gateway can make the device unreachable, this was not a comfortable development cycle.
We kept running into bugs. Not because the developers were careless, but because the tool was fighting us. Shell is great for small automation tasks. It is not great for implementing complex, stateful logic with many interacting conditions.
The Rewrite#
The C++ replacement took a fundamentally different approach. Instead of making hundreds of individual iptables calls, it built up the entire firewall configuration as a string — in the format that iptables-restore expects — and then fed it to iptables-restore in a single operation.
iptables-restore is designed to load a complete ruleset atomically. You give it a text file describing all your chains, policies, and rules, and it loads the whole thing into the kernel at once. One process invocation, one kernel interaction, done. Compared to hundreds of sequential iptables calls, the performance difference was dramatic.
The C++ application would:
- Query the central configuration server for the current device state (same message queue interface as before, but now called from C++ rather than shell)
- Based on that state, construct the complete iptables-restore formatted configuration using string manipulation
- Feed the result to
iptables-restore
The string building was not particularly elegant — it was a lot of appending lines to a string buffer based on conditionals — but it was straightforward C++ that could be debugged with normal tools, stepped through in a debugger, and reasoned about using normal programming patterns.
What Got Better#
Performance was the most visible improvement. The firewall configuration that used to take seconds now completed in a fraction of a second. On an embedded device where boot time matters and users notice delays when they change settings through the web UI, this was significant. The exact numbers are hazy at this point, but it was roughly an order of magnitude faster.
Reliability improved immediately. With iptables-restore, the ruleset is loaded atomically. Either the whole thing loads or none of it does. No more partially configured firewalls from a script that failed halfway through. If the configuration string had a syntax error, iptables-restore would reject it entirely and the previous ruleset would remain in place. That is a much better failure mode than “half your rules are applied and good luck figuring out which ones.”
Maintainability was the biggest long-term win. The firewall logic was now expressed in a real programming language with proper control flow, data structures, and error handling. New features that would have been a nightmare to implement in shell — things like dynamic rules that change based on VPN tunnel state, or complex port forwarding configurations — became straightforward.
We could also write tests. Not full integration tests against a real iptables stack, but we could test the string generation logic. Given this configuration state, does the code produce the correct iptables-restore formatted output? That kind of test was impossible with the shell script approach.
Debugging went from “add echo statements and pray” to “set a breakpoint and look at the state.” When a customer reported a firewall issue, we could reproduce their configuration, step through the code, and see exactly what rules were being generated and why.
What I Would Do Differently#
If I were doing this today, I would look more seriously at using nftables instead of iptables. The nftables framework is the successor to iptables in the Linux kernel, and it has a cleaner interface, better performance characteristics, and native support for atomic rule replacement without needing a separate restore tool. At the time of the rewrite, nftables was not mature enough for our needs, but that has changed.
I would also think harder about the configuration data model. We kept the same message queue interface to the configuration server, which meant we were still working with the same data representation as the shell script. If I were starting fresh, I would design the internal data model around the firewall’s actual needs rather than adapting to an interface that was designed for a shell script consumer.
The string building approach, while a big improvement over shell, is still a bit fragile. Building iptables-restore formatted text through string concatenation means typos in format strings can produce subtle bugs. A builder pattern or a proper abstraction over the rule format would have been worth the investment.
When to Make This Kind of Move#
Not every shell script should be rewritten in C++. That would be a waste of time in most cases. But there are signs that a shell script has outgrown its format:
- It is making hundreds of calls to the same external tool. If your script is just a loop around an external command, you are paying process invocation overhead for no reason. Look for a batch mode (like
iptables-restore) or a library interface. - The logic has too many interacting conditions. When your
ifstatements are nested four deep and depend on multiple pieces of state, you have a program, not a script. Programs deserve real programming languages. - You cannot test it. If the only way to verify a change is to run it on a live system, the feedback loop is too slow and the risk is too high.
- You keep finding bugs in the same area. Recurring bugs are a sign that the code is too hard to reason about correctly. Better tooling — types, a debugger, structured error handling — helps.
Shell is the right tool for plenty of jobs. But when you find yourself fighting it, that is not a skill issue. It is a signal that the problem has grown beyond what shell was designed to handle.
The firewall rewrite taught me that the best part of a performance optimization is often not the performance. It is all the other things that get better when you move to a tool that fits the problem.