Is it hard to port Frida on OpenWrt?

Frida “just works” on many common platforms:

Windows
Linux, glibc-based
Android
macOS and iOS

But what about porting it to your Linux-based home router… or any other Linux-powered appliance you might have on hand?

A little background: Frida can be used for user-space dynamic binary instrumentation and tracing, function call interception and modification, direct library call fuzzing, and other reverse engineering purposes.

The open-source toolkit runs locally on the target, and an agent, which includes a full JavaScript engine, is injected into a running process. The user then writes JavaScript applications to trace or hook function calls in any shared library the executable has mapped into its address space. Interaction with the target resident agent is performed programmatically via Python or JavaScript bindings, or standard tools such as frida-trace or the Frida CLI.

I set out to discover just how hard it is to make Frida run on an unsupported platform – and chronicle the challenges anyone might face when going through this process with another target.

Let’s find out together.

OpenWrt as an Example Target & Host Environment

For the purposes of facilitating writing this article and reproducibility, I pick a somewhat contrived target that illustrates what must be kept in mind during such a porting effort.

OpenWrt is a highly extensible GNU/Linux distribution primarily for network routers.
OpenWrt uses the musl C library run-time since version 17. However, 32-bit x86 with musl is an unsupported Frida configuration. Fortunately, for Frida testing purposes, it’s reasonably straightforward to spin up an x86 OpenWrt virtual machine.
OpenWrt uses BusyBox as the CLI environment and Dropbear as the SSH server. That is all in terms of regular Linux tools available from a default installation. For Frida integration with Python, I’d need to build quite a few packages from source code. To simplify the initial porting, since I’m only proving out viability to begin with, I decided to exclude Python support.
OpenWrt version 17 ships with an ancient musl C library version 1.1.x, and it appears the GCC used is from decades past. I would prefer to build the latest version of Frida targeting this platform. But, to avoid porting pains associated with compiling with an old version of GCC, I need to roll a custom GCC build with the target musl version (the current musl version isn’t ABI-compatible).

I select a common host environment for my tasks:

Ubuntu 22.04 with a slew of development tools installed
VMware Workstation for any virtualization needs

#FridaGoals

Here are the goals I had in my sights:

Build a suitable i686-linux-musl toolchain from a recent GCC code base, but with musl 1.1.x. (This should facilitate both compilation and run-time compatibility.) Frida binaries should be compatible with the on-target /lib/libc.so, and I won’t need to copy the libc.so build to the target and tinker with ELF RPATH sections.
Use the latest and greatest Frida version available, which is v16.5.6 at the time of this article’s publication.
Please note that since v16.2.x, the Frida build system has undergone a massive and wholesale refactoring. In general, the build system, old as well as new, is quite intricate and not for the faint of heart. All in all, the new build framework is an improvement, as no direct makefile, meson, or script hacking will be necessary for my new cross-compilation target.
Build the frida-inject tool (building the Python bindings and tools was beyond the scope of this effort). However, know that frida-inject includes all the really cool features, such as run-time process injection, JavaScript on-target run-time engine, tracing, and hooking. In summary, I built all the core features – but not the convenient Python interface.
Deploy the Frida agent into a running process on target and inject a simple JavaScript program that traces calls to a C library function. By default, there’s a web server for the administration interface, so I use this uhttpd daemon as the tracing target.

Creating an OpenWrt Virtual Machine

Let’s walk through this experiment, step by step.

First, create your OpenWrt VM using this release: https://downloads.openwrt.org/releases/17.01.2/targets/x86/generic/

From that page, download lede-17.01.2-x86-generic-combined-ext4.img.gz and use the following commands to convert it to VMDK format suitable for VMware Workstation.

user@jammy:~/dev/openwrt/convert$ ls
lede-17.01.2-x86-generic-combined-ext4.img.gz
user@jammy:~/dev/openwrt/convert$ gunzip lede-17.01.2-x86-generic-combined-ext4.img.gz
user@jammy:~/dev/openwrt/convert$ qemu-img convert -f raw -O vmdk lede-17.01.2-x86-generic-combined-ext4.img lede-17.01.2-x86-generic-combined-ext4.vmdk
user@jammy:~/dev/openwrt/convert$ ls
lede-17.01.2-x86-generic-combined-ext4.img  lede-17.01.2-x86-generic-combined-ext4.vmdk
user@jammy:~/dev/openwrt/convert$

Create a new virtual machine in VMware using the above file as “Use an existing virtual disk.” I used “Bridged network,” which assumes the network is served by a DHCP server.

Boot the OpenWrt virtual machine, and its console is automatically logged into as root user. (Since this VM is only intended for this exercise, we’re only interested in enabling one network interface for SSH and web server access.)

Via the console, use the vi editor and modify the /etc/config/network file to look like this:

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd81:8db8:a5bd::/48'

config interface 'lan'
	option ifname 'eth0'
	option proto 'dhcp'

The default configuration sets up a bridge, which we’re not interested in (for now).We just want eth0 to acquire an IP address from the regular DHCP server on the network.

After reboot, the web server and SSH daemon should be up and running. Verify the address on the console with the ifconfig command:

eth0  	Link encap:Ethernet  HWaddr 00:0C:29:64:DC:93  
      	inet addr:192.168.1.109  Bcast:192.168.1.255  Mask:255.255.255.0
      	inet6 addr: fe80::20c:29ff:fe64:dc93/64 Scope:Link
      	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      	RX packets:76 errors:0 dropped:0 overruns:0 frame:0
      	TX packets:84 errors:0 dropped:0 overruns:0 carrier:0
      	collisions:0 txqueuelen:1000
      	RX bytes:7788 (7.6 KiB)  TX bytes:8348 (8.1 KiB)
      	Interrupt:19 Base address:0x2000

Using your favorite browser to navigate to this IP address, the UI lets you change the password from blank to something else. Log in with a blank password and click Go to password configuration.

Depending on the host, you might see something like this when accessing the target with ssh:

user@jammy:~$ ssh root@192.168.1.109
Unable to negotiate with 192.168.1.109 port 22: no matching host key type found. Their offer: ssh-rsa
user@jammy:~$

Add the following to your client SSH configuration allowing RSA, and try again:

user@jammy:~$ cat .ssh/config
Host openwrt
  User root
  Hostname 192.168.1.109
  PubkeyAcceptedAlgorithms +ssh-rsa
  HostkeyAlgorithms +ssh-rsa
user@jammy:~$ ssh openwrt
root@192.168.1.109's password:


BusyBox v1.25.1 () built-in shell (ash)

 	_________
	/    	/\  	_	___ ___  ___
   /  LE	/  \	| |  | __|   \| __|
  /	DE  /	\   | |__| _|| |) | _|
 /________/  LE  \  |____|___|___/|___|                  	lede-project.org
 \    	\   DE /
  \	LE  \	/  -----------------------------------------------------------
   \  DE	\  /	Reboot (17.01.2, r3435-65eec8bd5f)
	\________\/	-----------------------------------------------------------

root@LEDE:~#

Inspecting the OpenWrt Target

Inspect the C runtime and the loader path first (that’s my preference, at least).

root@LEDE:/lib# ll /lib/ld-musl-i386.so.1
lrwxrwxrwx	1 root 	root         	7 Jun  8  2017 /lib/ld-musl-i386.so.1 -> libc.so*
root@LEDE:/lib# ./libc.so
musl libc (i386)
Version 1.1.16
Dynamic Program Loader
Usage: ./libc.so [options] [--] pathname [args]
root@LEDE:/lib# uname -a
Linux LEDE 4.4.71 #0 SMP Thu Jun 8 10:18:56 2017 i686 GNU/Linux
root@LEDE:/lib#

Later, we want to make sure the forthcoming toolchain generates ELF files with the loader path pointing to /lib/ld-musl-i386.so.1.

Now, verify the versions of libc.so and the Linux kernel; we’ll use this information in the next step.

Building a Toolchain (with a little help from OSS friends)

Two awesome OSS projects can help us craft a custom i686-linux-musl toolchain:

Both work, but let’s use mussel for this step because it’s very fast to build (thanks to an optimized and clever build order) – and makes it simple to select almost any version permutation of GCC, binutils, and musl.

Clone the mussel sources and make the following changes in the top script, per the versions found above.

user@jammy:~/dev/blog/pkg/mussel$ git diff
diff --git a/mussel b/mussel
index 23a3d61..b7ffe72 100755
--- a/mussel
+++ b/mussel
@@ -39,10 +39,10 @@ binutils_ver=2.42
 gcc_ver=13.2.0
 gmp_ver=6.3.0
 isl_ver=0.26
-linux_ver=6.5.3
+linux_ver=4.4.71
 mpc_ver=1.3.1
 mpfr_ver=4.2.1
-musl_ver=1.2.5
+musl_ver=1.1.16
 pkgconf_ver=2.1.0
 
 # ----- Package URLs ----- #
@@ -50,7 +50,7 @@ binutils_url=https://ftpmirror.gnu.org/binutils/binutils-$binutils_ver.tar.xz
 gcc_url=https://ftpmirror.gnu.org/gcc/gcc-$gcc_ver/gcc-$gcc_ver.tar.xz
 gmp_url=https://ftpmirror.gnu.org/gmp/gmp-$gmp_ver.tar.xz
 isl_url=https://libisl.sourceforge.io/isl-$isl_ver.tar.xz
-linux_url=https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-$linux_ver.tar.xz
+linux_url=https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-$linux_ver.tar.xz
 mpc_url=https://ftpmirror.gnu.org/mpc/mpc-$mpc_ver.tar.gz
 mpfr_url=https://www.mpfr.org/mpfr-current/mpfr-$mpfr_ver.tar.xz
 musl_url=https://musl.libc.org/releases/musl-$musl_ver.tar.gz
user@jammy:~/dev/blog/pkg/mussel$

Now the mussel script performs all the hard work:

user@jammy:~/dev/blog/pkg/mussel$ git clean -fdx
Removing builds/
Removing log.txt
Removing sources/
Removing toolchain/
user@jammy:~/dev/blog/pkg/mussel$ ./mussel i686 --parallel --enable-linux-headers

+======================================================+
|  mussel - The fastest musl libc Toolchain Generator  |
+------------------------------------------------------+
| 	Copyright (c) 2020-2024, Firas Khalil Khana  	|
|	Distributed under the terms of the ISC License	|
+======================================================+

Target Architecture:        	i686

Optional C++ Support:       	yes
Optional Fortran Support:   	no
Optional Linux Headers Support: yes
Optional OpenMP Support:    	no
Optional Parallel Support:  	yes
Optional pkg-config Support:	no
Optional Quadmath Support:  	no

.. Creating the sources directory...
.. Creating the builds directory...

.. Fetching binutils-2.42.tar.xz...
##################### 100.0% binutils-2.42.tar.xz##O=-                 	 
.. Verifying binutils-2.42.tar.xz...
binutils-2.42.tar.xz: OK
.. Unpacking binutils-2.42.tar.xz...
=> binutils-2.42.tar.xz prepared.

.. Fetching gcc-13.2.0.tar.xz...

Add mussel/toolchain/bin to the $PATH variable – perhaps in ~/.profile:

export PATH="$HOME/dev/blog/pkg/mussel/toolchain/bin:$PATH"

For a rudimentary test, we can compile something exceedingly simple:

user@jammy:~/dev/test$ which i686-linux-musl-gcc
/home/user/dev/blog/pkg/mussel/toolchain/bin/i686-linux-musl-gcc
user@jammy:~/dev/test$ cat main.c
#include <stdio.h>
#include <stdlib.h>
int
main() {
	printf( "hello\n" );
	exit(0);
}
user@jammy:~/dev/test$ i686-linux-musl-gcc -o main main.c
user@jammy:~/dev/test$ file main
main: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-i386.so.1, not stripped
user@jammy:~/dev/test$ scp main openwrt:/root
root@192.168.1.109's password:
main                                                                                               	100%   14KB   9.5MB/s   00:00    
user@jammy:~/dev/test$ readelf -d main

Dynamic section at offset 0x2efc contains 23 entries:
  Tag    	Type                     	Name/Value
 0x00000001 (NEEDED)                 	Shared library: [libc.so]
 0x0000000c (INIT)                   	0x1000

Confirm that the loader path is already set correctly, and the executable is dynamically linking against libc.so (if our loader path was incorrect, we could directly alter the ELF with the patchelf utility).

Because we selected the exact musl version in the mussel script, we can feel confident that the system libc and my compile-time libc are compatible. (Dealing with incompatible system- and compiler-provided libc is beyond the scope of this article.)

Run it:

root@LEDE:~# ./main
hello
root@LEDE:~#

This simple test is a promising sign that the custom toolchain is out-of-the-box compatible with the system-resident libc.

Cloning the Frida git Repository & Building Frida SDK

Clone the Frida repo (with –-recurse-submodules flag) from https://github.com/frida/frida.git.

Because Frida is actively maintained on the master branch, create a local branch off the 16.5.6 tag. Frida links with a number of libraries statically. So, first build the SDK with the toolchain, which includes all those dependencies. Frida supports two JavaScript engines, v8 and QuickJS. For now, exclude v8 (it’s much tricker to build in a cross-compile environment):

user@jammy:~/dev/blog/pkg/frida$ cat ../frida-sdk-conf.sh
#!/bin/bash

rm -rf deps build
git clean -fdx
git submodule foreach --recursive git clean -xfd
git submodule foreach --recursive git checkout .

./releng/deps.py build --exclude v8 --bundle=sdk --host=i686-linux-musl
user@jammy:~/dev/blog/pkg/frida$ ../frida-sdk-conf.sh
Downloading toolchain 20240913...
Extracting toolchain...

╭────
│ 📦 zlib
├───────────────────────────────────────────────╮
│ URL: https://github.com/frida/zlib.git
│ CID: a912d314d0812518d4bbd715a981e6c9484b550d
├───────────────────────────────────────────────╯
│ zlib :: Cloning
│ zlib :: Building for linux-x86-musl

Once finished, find the SDK under deps:

│ libsoup :: Building for linux-x86-musl

╭────
│ 🏗️  Packaging
├───────────────────────────────────────────────╮
│ sdk-linux-x86-musl.tar.xz :: Staging files
│ sdk-linux-x86-musl.tar.xz :: Assembling
│ sdk-linux-x86-musl.tar.xz :: All done

╭────
│ 🎉 Done
├───────────────────────────────────────────────╮
  	Total: 00:06:31
	Prepare: 00:00:01
  	Clone: 00:01:02
  	Build: 00:04:49
  Packaging: 00:00:38

user@jammy:~/dev/blog/pkg/frida$ ll deps
total 36116
drwxrwxr-x  4 user user 	4096 Nov 13 18:39 ./
drwxrwxr-x  8 user user 	4096 Nov 13 18:33 ../
-rw-rw-r--  1 user user 36959396 Nov 13 18:39 sdk-linux-x86-musl.tar.xz
drwxrwxr-x 31 user user 	4096 Nov 13 18:39 src/
drwx------  5 user user 	4096 Sep 13 08:46 toolchain-linux-x86_64/
user@jammy:~/dev/blog/pkg/frida$

We won’t really need to bother with the SDK from this point on; the Frida build picks it up when it needs it.

Building Frida – First Attempt

Build Frida without the “tools” or Python support:

user@jammy:~/dev/blog/pkg/frida$ ./configure \
	--prefix=/home/user/dev/blog/pkg/local   \
	--enable-gadget --enable-server      	\
	--enable-portal --enable-inject      	\
	--host=i686-linux-musl
Downloading toolchain 20241006...
Extracting toolchain...
Downloading SDK 20241006 for linux-x86_64...
Extracting SDK...
Deploying local SDK...
The Meson build system
Version: 1.4.99
Source dir: /home/user/dev/blog/pkg/frida
Build dir: /home/user/dev/blog/pkg/frida/build
Build type: cross build
Project name: frida
Project version: 16.5.6

Note the final output from the configuration step:

frida-core| Subproject frida-core finished.

Build targets in project: 44

frida-core 16.5.6

  Backends
	local        	: YES
	fruity       	: NO
	droidy       	: NO
	socket       	: YES
	barebone     	: NO
	compiler     	: YES

  Features
	compat       	: enabled by default for linux-x86
	assets       	: embedded
	mapper       	: NO
	connectivity 	: YES
	compiler_snapshot: NO
	gadget       	: YES
	server       	: YES
	portal       	: YES
	inject       	: YES
	tests        	: NO

frida 16.5.6

  Subprojects (for host machine)
	frida-core 	: YES
	frida-gum  	: YES

  User defined options
	Cross files	: /home/user/dev/blog/pkg/frida/build/frida-linux-x86-musl.txt
	Native files   : /home/user/dev/blog/pkg/frida/build/frida-linux-x86_64.txt
	default_library: static
	optimization   : s
	prefix     	: /home/user/dev/blog/pkg/local
	strip      	: true
	b_ndebug   	: true
	gadget     	: enabled
	inject     	: enabled
	portal     	: enabled
	server     	: enabled

Found ninja-1.11.1 at /home/user/dev/blog/pkg/frida/deps/toolchain-linux-x86_64/bin/ninja
user@jammy:~/dev/blog/pkg/frida$

We’re building the frida-core package with the frida-inject tool. The Cross files line points to a meson toolchain configuration file used throughout the build. For good measure, visually inspect it for discrepancies. Choose a local path as the installation prefix, and use that for subsequent builds below.

However, the build fails with:

user@jammy:~/dev/blog/pkg/frida$ make
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /home/user/dev/blog/pkg/frida/deps/toolchain-linux-x86_64/bin/ninja
[190/242] Linking target subprojects/frida-core/lib/gadget/libfrida-gadget-raw.so
FAILED: subprojects/frida-core/lib/gadget/libfrida-gadget-raw.so
/home/user/dev/blog/pkg/mussel/toolchain/bin/../lib/gcc/i686-linux-musl/13.2.0/../../../../i686-linux-musl/bin/ld: /home/user/dev/blog/pkg/frida/deps/sdk-linux-x86-musl/lib/libunwind.a(x86_Los-linux.c.o): in function `_ULx86_local_resume':
/home/user/dev/blog/pkg/frida/deps/src/_sdk.tmp/linux-x86-musl/libunwind/../../../libunwind/src/x86/Los-linux.c:309:(.text._ULx86_local_resume+0x42): undefined reference to `setcontext'
collect2: error: ld returned 1 exit status

Why? Because standard glibc includes a lot of extra support functions not specified by Posix, and musl strives to be as compliant as possible. Fortunately, there are OSS libraries that provide the ucontext and iconv functionality needed for a clean build.

Building Support Libraries

Build the necessary libraries, as identified in the previous step:

https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.17.tar.gz: Download version 1.17
https://github.com/kaniini/libucontext/archive/refs/tags/libucontext-1.3.2.tar.gz: Download version 1.3.2

Starting with libiconv; download, untar, configure, build and install:

user@jammy:~/dev/blog/pkg$ tar xf libiconv-1.17.tar.gz
user@jammy:~/dev/blog/pkg$ cd libiconv-1.17/
user@jammy:~/dev/blog/pkg/libiconv-1.17$ cat ../libiconv-conf.sh
#!/bin/bash

./configure --host=i686-linux-musl    	\
	--enable-relocatable --enable-static  \
	--prefix=/home/user/dev/blog/pkg/local
make CFLAGS="-fPIC"
make install
user@jammy:~/dev/blog/pkg/libiconv-1.17$ ../libiconv-conf.sh
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for i686-linux-musl-strip... i686-linux-musl-strip
checking for a race-free mkdir -p... /usr/bin/mkdir -p

And build and install libucontext as well:

user@jammy:~/dev/blog/pkg$ tar xf libucontext-libucontext-1.3.2.tar.gz
user@jammy:~/dev/blog/pkg$ cd libucontext-libucontext-1.3.2/
user@jammy:~/dev/blog/pkg/libucontext-libucontext-1.3.2$ cat ../libucontext-conf.sh
#!/bin/bash

make ARCH=x86 CC=i686-linux-musl-gcc prefix=/home/user/dev/blog/pkg/local
make ARCH=x86 CC=i686-linux-musl-gcc prefix=/home/user/dev/blog/pkg/local install
user@jammy:~/dev/blog/pkg/libucontext-libucontext-1.3.2$ ../libucontext-conf.sh
i686-linux-musl-gcc -std=gnu99 -D_DEFAULT_SOURCE -fPIC -DPIC -ggdb3 -O2 -Wall -Iinclude -Iarch/x86 -Iarch/common -Iarch/common/include -DFORCE_SOFT_FLOAT -DEXPORT_UNPREFIXED -c -o arch/x86/makecontext.o arch/x86/makecontext.c

Because the libraries are configured with prefix=, the make install commands copy the relevant files into this folder.

Building Frida – Second Attempt (the pressure’s on)

Repeat the same configure, make, and make install sequence as above. But, in the configure step, add a meson line to include the additional libraries to all linkage commands:

user@jammy:~/dev/blog/pkg/frida$ cat ../frida-conf.sh
#!/bin/bash
set -x

LOCPATH=$HOME/dev/blog/pkg/local/lib
FRICORE=subprojects/frida-core
HLPRS=${FRICORE}/src/linux/helpers

rm -rf build/
git submodule foreach --recursive git clean -xfd

./configure --prefix=/home/user/dev/blog/pkg/local --enable-gadget \
  --enable-server --enable-portal --enable-inject --host=i686-linux-musl \
  -- -Dc_link_args="${LOCPATH}/libucontext_posix.a ${LOCPATH}/libucontext.a ${LOCPATH}/libiconv.a"

make
make install
user@jammy:~/dev/blog/pkg/frida$ ../frida-conf.sh

This time, the build should complete cleanly. Look for the frida-inject binary in the local/bin folder:

user@jammy:~/dev/blog/pkg/frida$ ll ../local/bin/
total 96820
drwxrwxr-x 2 user user 	4096 Nov 13 19:11 ./
drwxrwxr-x 6 user user 	4096 Sep 17 17:05 ../
-rwxr-xr-x 1 user user 32974124 Nov 13 19:11 frida-inject*
-rwxr-xr-x 1 user user 33076492 Nov 13 19:11 frida-portal*
-rwxr-xr-x 1 user user 33023276 Nov 13 19:11 frida-server*
-rwxr-xr-x 1 user user	49404 Nov 13 18:58 iconv*
user@jammy:~/dev/blog/pkg/frida$

A Note on frida-inject

Though frida-inject looks like just one giant executable, for ease of use, the following discrete pieces of code are embedded into the binary and are spliced out during different stages of target process hijacking:

bootstrapper: After frida-inject has seized control of the victim process with the ptrace system call, it runs this discrete executable in the victim context and it resolves a minimum set of libc symbols. Note that Frida also supports the much more complicated case where the victim process is using an unknown or custom libc, and the Frida tools are using a copied libc from my custom toolchain.
loader: This also runs in the victim context, and by using the symbols resolved by the bootstrapper, it makes use of some shared memory, with which frida-inject transfers the agent to the victim process. Finally, the loader calls dlopen() on the agent shared library residing in the shared memory, creates a thread, and launches the agent main thread.
agent: The agent is a shared library, which makes it convenient for the above loader to map it into the victim address space. Once loaded and started, the agent establishes communication with frida-inject. Then the JavaScript tracing application is transferred to the agent, which executes it and connects stdout back to frida-inject.

Running Frida – First Attempt (here we go!)

frida-inject is monolith; its only dependency is regular libc. Therefore, you only need to SCP this executable to the target. Transfer a super-simple JavaScript file to send to the agent for execution in the victim process context:

root@LEDE:~# cat js/read-hook.js
 var readfun = Module.findExportByName("libc.so", "read")
 Interceptor.attach(readfun, {
  onEnter: function (args, state) {
     	console.log("[+] CALLED READ");
     	console.log('read called:\n' +
    	Thread.backtrace(this.context, Backtracer.ACCURATE)
    	.map(DebugSymbol.fromAddress).join('\n') + '\n');
  },
  onLeave: function (retval) {
  }
 });
root@LEDE:~#

The script basically just traces all calls to the standard libc read function:

root@LEDE:~# ./frida-inject --help
Usage:
  frida-inject [OPTION*]

Help Options:
  -h, --help                       	Show help options

Application Options:
  -D, --device=ID                  	connect to device with the given ID
  -f, --file=FILE                  	spawn FILE
  -p, --pid=PID                    	attach to PID
  -n, --name=NAME                  	attach to NAME
  -r, --realm=REALM                	attach in REALM
  -s, --script=JAVASCRIPT_FILENAME	 
  -R, --runtime=qjs|v8             	Script runtime to use
  -P, --parameters=PARAMETERS_JSON 	Parameters as JSON, same as Gadget
  -e, --eternalize                 	Eternalize script and exit
  -i, --interactive                	Interact with script through stdin
  --development                    	Enable development mode
  --version                        	Output version information and exit

root@LEDE:~#

Typically I use the -n or -p in conjunction with -s:

root@LEDE:~# ./frida-inject -n uhttpd -s js/read-hook.js
Unexpected failure while trying to allocate memory
root@LEDE:~#

Unfortunately, it fails the first attempt. Troubleshooting these sorts of errors is extremely time-consuming. Why? Because code is running both in the hijacked target process and on the client side as frida-inject. And there’s synchronization and communication between the two. Another complicating factor is that most of the logic in Frida is implemented in the Vala language (see https://vala.dev/).

Without elaborating on the painstaking troubleshooting effort figuring out this run-time error, suffice to say, it turns out that there are two problems:

The bootstrapper makes some assumptions about the expected names and paths for the loader name present in the ELF files, which aren’t necessarily true on a musl platform.
Frida-inject gains control of the victim process with a ptrace system call, and it wants, in most situations, pivot execution to call mmap on an anonymous file by explicitly setting instruction pointer and register values. It appears this might not work correctly in all situations on x86 32-bit. We need to force the code path to run the backup implementation for code execution, which basically involves temporarily “borrowing” a piece of a code segment.

The following patch should be applied onto subprojects/frida-core:

diff --git a/src/linux/frida-helper-backend.vala b/src/linux/frida-helper-backend.vala
index 9da2152..cca135c 100644
--- a/src/linux/frida-helper-backend.vala
+++ b/src/linux/frida-helper-backend.vala
@@ -1083,7 +1083,7 @@ namespace Frida {
 			uint64 remote_munmap = 0;
 			ProcMapsEntry? remote_libc = ProcMapsEntry.find_by_path (pid, local_libc.path);
 			bool same_libc = remote_libc != null && remote_libc.identity == local_libc.identity;
-			if (same_libc) {
+			if (false) {
 				remote_mmap = remote_libc.base_address + mmap_offset;
 				remote_munmap = remote_libc.base_address + munmap_offset;
 			}
diff --git a/src/linux/helpers/Makefile b/src/linux/helpers/Makefile
index f81563e..b593bf8 100644
--- a/src/linux/helpers/Makefile
+++ b/src/linux/helpers/Makefile
@@ -1,4 +1,4 @@
-top_srcdir := ../../..
+top_srcdir := ../../../..
 releng := $(top_srcdir)/releng
 
 BUILDDIR ?= $(top_srcdir)/build
@@ -7,19 +7,20 @@ FRIDA_MONOREPO ?= $(top_srcdir)/..
 ifdef FRIDA_HOST
 host_os := $(shell echo $(FRIDA_HOST) | cut -f1 -d"-")
 host_arch := $(shell echo $(FRIDA_HOST) | cut -f2 -d"-")
+host_variant := $(shell echo $(FRIDA_HOST) | cut -f3 -d"-")
 else
 host_os := $(shell $(releng)/detect-os.sh)
 host_arch := $(shell $(releng)/detect-arch.sh)
 endif
 
 crossfile := $(shell test -d "$(BUILDDIR)" \
-		&& echo "$(BUILDDIR)/frida-$(host_os)-$(host_arch).txt" \
-		|| echo $(FRIDA_MONOREPO)/build/frida*-$(host_os)-$(host_arch).txt)
+		&& echo "$(BUILDDIR)/frida-$(host_os)-$(host_arch)-$(host_variant).txt" \
+		|| echo $(FRIDA_MONOREPO)/build/frida*-$(host_os)-$(host_arch)-$(host_variant).txt)
 
 build: ext/linux/tools/include/nolibc/nolibc.h
 	rm -rf build
 	meson setup --cross-file $(crossfile) -Db_lto=true build
-	meson compile -C build
+	meson compile -C build -v
 	cp build/bootstrapper.bin bootstrapper-$(host_arch).bin
 	cp build/loader.bin loader-$(host_arch).bin
 
diff --git a/src/linux/helpers/bootstrapper.c b/src/linux/helpers/bootstrapper.c
index 5f579ae..3fabc6f 100644
--- a/src/linux/helpers/bootstrapper.c
+++ b/src/linux/helpers/bootstrapper.c
@@ -550,7 +550,9 @@ frida_infer_rtld_flavor_from_filename (const char * name)
   if (frida_str_has_prefix (name, "ld-uClibc"))
     return FRIDA_RTLD_UCLIBC;
 
-  if (strcmp (name, "libc.so") == 0)
+  if (strcmp (name, "libc.so") == 0 ||
+      frida_str_has_prefix (name, "libc.musl") ||
+      frida_str_has_prefix (name, "ld-musl"))
     return FRIDA_RTLD_MUSL;
 
   if (strcmp (name, "ld-android.so") == 0)
@@ -607,7 +609,9 @@ frida_path_is_libc (const char * path, FridaRtldFlavor rtld_flavor)
   else
     name = path;
 
-  return frida_str_has_prefix (name, "libc.so");
+  return frida_str_has_prefix (name, "libc.so") ||
+  			frida_str_has_prefix (name, "ld-musl") ||
+  			frida_str_has_prefix (name, "libc.musl");
 }
 
 static ssize_t

Building Frida – Third Attempt (c’mon now…)

Delete the build folder and clean out the submodule. However, since we want to retain the SDK build, avoid cleaning the top-level frida directory. Run the same configuration step as above. Rebuild the bootstrapper and loader explicitly, because they’re normally not rebuilt. Kick off the top-level build by invoking make and make install:

user@jammy:~/dev/blog/pkg/frida$ cat ../frida-conf.sh
#!/bin/bash
set -x

LOCPATH=$HOME/dev/blog/pkg/local/lib
FRICORE=subprojects/frida-core
HLPRS=${FRICORE}/src/linux/helpers

rm -rf build/
git submodule foreach --recursive git clean -xfd
( cd ${FRICORE} && git checkout . )

./configure --prefix=/home/user/dev/blog/pkg/local --enable-gadget \
  --enable-server --enable-portal --enable-inject --host=i686-linux-musl \
  -- -Dc_link_args="${LOCPATH}/libucontext_posix.a ${LOCPATH}/libucontext.a ${LOCPATH}/libiconv.a"

( cd ${FRICORE} && patch -p 1 < ../../../frida-core.patch )
( cd ${HLPRS} && make FRIDA_HOST=linux-x86-musl )

make
make install
user@jammy:~/dev/blog/pkg/frida$ ../frida-conf.sh

Running Frida – Second Attempt (success!)

Transfer the new frida-inject to the target and repeat the command. If everything goes well, it shouldn’t return to the console.

Start a second SSH session, and verify the agent shared library has been memory mapped into the process:

root@LEDE:~# ps | grep uhttpd
 1953 root 	20624 S	/usr/sbin/uhttpd -f -h /www -r LEDE -x /cgi-bin -u /ubus -t 60 -T 30 -k 20 -A 1 -n 3 -N 100 -R -p 0.0.0.0:80 -p [::]:80
 2325 root 	33760 S	./frida-inject -n uhttpd -s js/read-hook.js
 2354 root   	956 R	grep uhttpd
root@LEDE:~# cat /proc/1953/maps
08048000-08052000 r-xp 00000000 08:02 1042   	/usr/sbin/uhttpd
08052000-08053000 r--p 00009000 08:02 1042   	/usr/sbin/uhttpd
08053000-08054000 rw-p 0000a000 08:02 1042   	/usr/sbin/uhttpd
08054000-08057000 rw-p 00000000 00:00 0
0995d000-09972000 rw-p 00000000 00:00 0      	[heap]
b63c4000-b6427000 r--p 00000000 08:02 291    	/lib/libc.so
b6427000-b6432000 r--p 00000000 08:02 1042   	/usr/sbin/uhttpd
b6432000-b643c000 r--p 00000000 08:02 296    	/lib/libubox.so
b6604000-b66ea000 r--p 00000000 00:05 999    	/memfd:frida-agent-32.so (deleted)
b66ea000-b6d9d000 r-xp 000e6000 00:05 999    	/memfd:frida-agent-32.so (deleted)
b6d9d000-b75d2000 r--p 00799000 00:05 999    	/memfd:frida-agent-32.so (deleted)
b75d2000-b76ba000 r--p 00fce000 00:05 999    	/memfd:frida-agent-32.so (deleted)
b76ba000-b76c3000 rw-p 010b6000 00:05 999    	/memfd:frida-agent-32.so (deleted)

Finally, launch a browser and navigate to the OpenWrt IP address:

root@LEDE:~# ./frida-inject -n uhttpd -s js/read-hook.js
[+] CALLED READ
read called:
0xb7733060 libubox.so!ustream_printf+0x1c1
0xb7731ca4 libubox.so!uloop_run+0x31c
0x804a384 uhttpd!__register_frame_info_bases+0x9c4
0xb77547ee libc.so!__libc_start_main+0x29
0x804a3e3 uhttpd!__register_frame_info_bases+0xa23
0x804a3be uhttpd!__register_frame_info_bases+0x9fe

[+] CALLED READ
read called:
0xb7733060 libubox.so!ustream_printf+0x1c1
0xb7731ca4 libubox.so!uloop_run+0x31c
0x804a384 uhttpd!__register_frame_info_bases+0x9c4
0xb77547ee libc.so!__libc_start_main+0x29
0x804a3e3 uhttpd!__register_frame_info_bases+0xa23
0x804a3be uhttpd!__register_frame_info_bases+0x9fe

It works!

Naturally, the read trace script should trigger after you hit the web server from the client browser. Not the most exciting script, but this simple example shows the entire sequence of hijacking, injection, on-target agent launch, and JavaScript execution functions correctly. You can write much more intriguing scripts quite easily, such as:

Print backtrace to whomever called the traced function
Modify parameters upon call interception
Explicitly call functions without trace hooks

But the important thing – we did it.

For reference, I submitted pull requests for the bootstrapper and makefile patches to the frida-core project repository at https://github.com/frida/frida-core, which have already been accepted into mainline. I’d love to investigate why the mmap pivot trick appears to fail on this x86 32-bit platform and provide the feedback to the Frida project, but that looks to be an excruciating and time-consuming endeavor. That said, I did submit a github issue for tracking purposes.

The Thrill of Success (and yes, there’s a Part II)

Well, the experience of porting Frida to an unsupported Linux platform felt rewarding. Porting Frida to an unsupported platform – troubleshooting included – was a smidge harder than I expected, but far from impossible. Even the most seasoned engineer would be hard-pressed to complete this in a day, start to finish. YMMV, but for me, it was roughly a week-long effort.

Though I walked us through the steps I took for OpenWrt, yours may diverge based on how uncommon and exotic the platform is. Plus, potential problems vary with the C library used. While I explored a musl-based platform, different problems could emerge with newlib, uclibc, etc. (which are common among small-footprint Linux appliances).

Have you ever taken Frida outside its normal swim lane? Let us know about your prior experience – or, perhaps sharing our knowledge will inspire you to dive into a Frida-porting project this weekend.

If people are excited about this topic, I’ll write a follow-up article [edit: my Part II is now live!]. Part II would take the Frida port a few steps further, adding additional run-time libraries and tools:

Latest Python 3.12 or 3.13 with the packages necessary for Frida integration
A few libraries necessary for comprehensive Python support
Frida Python-based tools, such as frida-trace
Fpicker tool, which is a simple and straightforward fuzzing framework written in JavaScript