I have a dump of one of the flash partitions of my recently purchased Zyxel GS1900-24HPv2. With that in hand, I will be able to continue the analysis of how this switch works. Will this JFFS2 filesystem image contain the missing scripts that are needed for Linux to boot on this switch?
At the end of the previous article, Dumping the Flash, I had successfully dumped the /dev/mtdblock3 to an image file. In theory, this is just a JFFS2 filesystem image that can be mounted and checked out. Once mounted, I will be able to see the contents of /mnt/startup-config.
Hopefully this file will offer the next clues to how this firmware works.
To poke around at this filesystem I will once again turn to binwalk, and introduce a second tool called Jefferson.
Let's take a quick look at what is in the image.
binwalk -B jffs2_cfg.img
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
0 0x0 JFFS2 filesystem, big endian
500 0x1F4 Zlib compressed data, compressed
2004 0x7D4 Zlib compressed data, compressed
2544 0x9F0 Zlib compressed data, compressed
3284 0xCD4 Zlib compressed data, compressed
3976 0xF88 Zlib compressed data, compressed
5488 0x1570 Zlib compressed data, compressed
6540 0x198C Zlib compressed data, compressed
7144 0x1BE8 Zlib compressed data, compressed
7332 0x1CA4 Zlib compressed data, compressed
7628 0x1DCC Zlib compressed data, compressed
8436 0x20F4 Zlib compressed data, compressed
9248 0x2420 Zlib compressed data, compressed
10112 0x2780 Zlib compressed data, compressed
10864 0x2A70 Zlib compressed data, compressed
11616 0x2D60 Zlib compressed data, compressed
12480 0x30C0 Zlib compressed data, compressed
13212 0x339C Zlib compressed data, compressed
13944 0x3678 Zlib compressed data, compressed
14788 0x39C4 Zlib compressed data, compressed
15528 0x3CA8 Zlib compressed data, compressed
16268 0x3F8C Zlib compressed data, compressed
17120 0x42E0 Zlib compressed data, compressed
17860 0x45C4 Zlib compressed data, compressed
18600 0x48A8 Zlib compressed data, compressed
19452 0x4BFC Zlib compressed data, compressed
20192 0x4EE0 Zlib compressed data, compressed
20932 0x51C4 Zlib compressed data, compressed
21648 0x5490 JFFS2 filesystem, big endian
22324 0x5734 JFFS2 filesystem, big endian
29756 0x743C Zlib compressed data, compressed
30428 0x76DC JFFS2 filesystem, big endian
30552 0x7758 Zlib compressed data, compressed
31224 0x79F8 JFFS2 filesystem, big endian
31352 0x7A78 Zlib compressed data, compressed
32204 0x7DCC Zlib compressed data, compressed
32944 0x80B0 Zlib compressed data, compressed
33684 0x8394 Zlib compressed data, compressed
34536 0x86E8 Zlib compressed data, compressed
35276 0x89CC Zlib compressed data, compressed
36016 0x8CB0 Zlib compressed data, compressed
36868 0x9004 Zlib compressed data, compressed
37608 0x92E8 Zlib compressed data, compressed
38348 0x95CC Zlib compressed data, compressed
39200 0x9920 Zlib compressed data, compressed
39940 0x9C04 Zlib compressed data, compressed
40680 0x9EE8 Zlib compressed data, compressed
41532 0xA23C Zlib compressed data, compressed
42272 0xA520 Zlib compressed data, compressed
43012 0xA804 Zlib compressed data, compressed
43864 0xAB58 Zlib compressed data, compressed
44604 0xAE3C Zlib compressed data, compressed
45344 0xB120 Zlib compressed data, compressed
46196 0xB474 Zlib compressed data, compressed
46936 0xB758 Zlib compressed data, compressed
47676 0xBA3C Zlib compressed data, compressed
48528 0xBD90 Zlib compressed data, compressed
49268 0xC074 Zlib compressed data, compressed
50008 0xC358 Zlib compressed data, compressed
50860 0xC6AC Zlib compressed data, compressed
51600 0xC990 Zlib compressed data, compressed
52340 0xCC74 Zlib compressed data, compressed
53192 0xCFC8 Zlib compressed data, compressed
53932 0xD2AC Zlib compressed data, compressed
54672 0xD590 Zlib compressed data, compressed
55524 0xD8E4 Zlib compressed data, compressed
56196 0xDB84 JFFS2 filesystem, big endian
This is crazy! There are several JFFS2 filesystems, along with quite a few Zlib compressed sections of data. I wonder what shows up with strings.
The output of this is really long so I'm only including some snippets here.
strings jffs2_cfg.img
Vssh
Xssh_host_rsa_v2_key
ssh_host_rsa_v2_key.pub
ssh_host_dsa_v2_key
ssl_key.pem
ssl_cert.pem
http_rsa_key
startup-config
backup-config
tmp-startup-config
ssh_host_dsa_v2_key
ssh_host_rsa_v2_key.pubFrom here, it goes on and on with more SSH keys and more references to startup-config. I am sort of interested in knowing why there are so many SSH keys, but am also pumped that startup-config is showing up!
Now, how do I actually get to the data? There are two approaches that I want to explore.
Realistically, either of these options should have the same effect: the JFFS2 image will show up as files on my computer. They can then be browsed using either a terminal or file manager like any other files on my computer.
binwalk can be used to extract the files in the same way that we did with the SquashFS image in the first article, Exploring Firmware of a Zyxel Switch. Doing so requires a second program called Jefferson.
We can mount the JFFS2 image just like any other image or drive. This will require a few kernel modules to be installed.
Extracting the content of the image is pretty straightforward. We are able to use binwalk with the extract and Matryoshka flags. But there is a catch - binwalk needs to use a second program called Jefferson to do the extraction.
How to install Jefferson will vary depending on what system you are on. I am using an Arch workstation and was able to find it in the AUR. I was not, however, able to build the version in the AUR. I ended up going against Arch guidelines and installing it with PIP. Maybe by the time you are reading this the AUR version will build again, or you are on a different system that is a bit simpler.
With Jefferson installed, we can just do:
binwalk -eM jffs2_cfg.img
ls _jffs2_cfg.img.extracted
0.jffs2 1B04 5174.jffs2 6B18 6F8C.jffs2 940.zlib jffs2-root-0 jffs2-root-3 jffs2-root-6
1A48 1B04.zlib 649C.jffs2 6B18.zlib 7298.jffs2 D074.jffs2 jffs2-root-1 jffs2-root-4 jffs2-root-7
1A48.zlib 4F10.jffs2 688C.jffs2 6B90.jffs2 940 jffs2-root jffs2-root-2 jffs2-root-5This ends up extracting several JFFS2 filesystems from the single image. There are no less than 6 filesystems that are extracted and ready to be viewed. Exciting!
Now to just take a look and see what we have.
ls _jffs2_cfg.img.extracted/jffs2-root -Al
backup-config ssh tmp-startup-config
ls _jffs2_cfg.img.extracted/jffs2-root-0 -Al
backup-config ssh tmp-startup-config
ls _jffs2_cfg.img.extracted/jffs2-root-1 -Al
backup-config ssh tmp-startup-config
ls _jffs2_cfg.img.extracted/jffs2-root-2 -Al
backup-config tmp-startup-config
ls _jffs2_cfg.img.extracted/jffs2-root-3 -Al
tmp-startup-config
...
Why are there so many duplicates?
This is part of how JFFS2, or Journalling Flash Filesystem Version 2, works. When you create a file, a new journal node is created that describes the location of this file and the name. If you then delete the file, another new node is created that indicates that the file has been deleted. The original file creation node still exists. This can be marked as obsolete later and removed, but it isn't necessarily going to happen instantly.
When a JFFS2 partition is mounted the full journal needs to be read. The filesystem is recreated based on the entries in the journal. What you would see on the target system is the end result of any additions, subtractions, and changes to the files that were originally there.
Even though we see multiple instances of the ssh directory, tmp-startup-config, and backup-config, these are most likely just changes that have happened to those objects over time. Suspiciously, startup-config is missing completely.
This has definitely moved me closer to the goal, but it seems like I might be on a witch hunt.
It is actually rather hard to find any information about how to do this that is less than 10 years old. And it seems like the old information is a bit wrong.
At least three file systems have been developed as JFFS2 replacements: LogFS, UBIFS, and YAFFS.
JFFS2 was introduced in 2001 and successors were created not long after that. Most of the information that I could find online seems to be within that time range.
We need to make sure a few kernel modules are present.
Most articles online also reference mtdchar. This was needed before kernel v3.9 but has since been merged into mtdcore.
When we initialize the mtdram kernel module we need to pass the total size of the image in kilobytes. In this case it is super easy because it is exactly 1024 KB. However, to calculate what to use on other images:
du -sk jffs2_cfg.imgWe can then use the result of that while initializing the image.
modprobe mtdram total_size=[IMAGE_SIZE_IN_KB] erase_size=128
modprobe mtdblock
mkdir mnt
dd if=jffs2_cfg.img of=/dev/mtdblock0
mount -t jffs2 /dev/mtdblock0 mnt
mount: /home/vivek/Projects/zyxel/docs/mnt: can't read superblock on /dev/mtdblock0.
dmesg(1) may have more information after failed mount system call.Come on. CaN't ReAd SuPeRbLoCk.
dmesg | tail -n 200
[604946.026542] jffs2: version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc.
[604946.026844] jffs2: Magic bitmask is backwards at offset 0x00000000. Wrong endian filesystem?
[604946.026848] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000008: 0x60f0 instead
[604946.026856] jffs2: Further such events for this erase block will not be printed
[604946.026870] jffs2: Old JFFS2 bitmask found at 0x00001614
[604946.026871] jffs2: You cannot use older JFFS2 filesystems with newer kernels
[604946.027005] jffs2: Empty flash at 0x0000e140 ends at 0x00010000At least this is helpful. I'm using an x86 workstation, which is little-endian. The MIPS processor in this switch is big-endian. (Endianness) Maybe this kernel driver will only work with the same endianness of the host?
This image is also apparently an older JFFS2 filesystem. I was not aware that older JFFS2 filesystems were not able to be used with newer kernels.
Luckily, the mtd-utils package comes with a tool called jffs2dump. We are able to pretty easily convert the big endian image to little endian.
jffs2dump -r -e jffs2_le.img -b jffs2_cfg.imgThis threw a bunch of warnings but did create the new image. Let's see if it can be mounted.
sudo dd if=jffs2_le.img of=/dev/mtdblock0
[sudo] password for vivek:
1951+1 records in
1951+1 records out
999180 bytes (999 kB, 976 KiB) copied, 0.0037887 s, 264 MB/s
sudo mount -t jffs2 /dev/mtdblock0 mntWell... it didn't throw any errors!
ls mnt -Al
total 3
-rw-r--r-- 1 root root 1293 Dec 31 2021 backup-config
drwxr-xr-x 2 root root 0 Dec 31 2021 ssh
-rw-r--r-- 1 root root 1293 Dec 31 2021 tmp-startup-config
Alright, so this has been an epic journey so far. I am hunting for a file called startup-config but it does not want to be found. startup-config's brother and sister are around, but he just noped out. Dude just skipped town and went into hiding.
What can tmp-startup-config tell us?
cat mnt/tmp-startup-config
! System Description: ZyXEL GS1900-24HP Switch
! Revision: B1
! Serial Number: S222L31002671
! MAC Address Range: D8:EC:E5:BB:5E:08 - D8:EC:E5:BB:5E:22
! Boot Version: V2.0.2.5 | 08/31/2021
! Firmware Version: V2.80(ABTP.0) | 01/11/2022
! System Up Time: 0 days, 0 hours, 1 mins, 52 secs
!
!
!
ip address 192.168.1.1 mask 255.255.255.0
username "admin" secret 8 $8$hytCmvXU$dfcb8b69b124f7b794afd232bfdf2d0aa1d92a7c6229b52f7401f177a1201487
voice-vlan oui-table 00:E0:BB "3COM"
voice-vlan oui-table 00:03:6B "Cisco"
voice-vlan oui-table 00:E0:75 "Veritel"
voice-vlan oui-table 00:D0:1E "Pingtel"
voice-vlan oui-table 00:01:E3 "Siemens"
voice-vlan oui-table 00:60:B9 "NEC/Philips"
voice-vlan oui-table 00:0F:E2 "H3C"
voice-vlan oui-table 00:09:6E "Avaya"
!
!
!
!
!
spanning-tree mst configuration
name "D8:EC:E5:BB:5E:08"
!
!
!
snmp community "public" rw
!
!
no ip ssh
!
!
management access-list default
!
interface 1
!
interface 2
!
interface 3
!
interface 4
!
interface 5
!
interface 6
!
interface 7
!
interface 8
!
interface 9
!
interface 10
!
interface 11
!
interface 12
!
interface 13
!
interface 14
!
interface 15
!
interface 16
!
interface 17
!
interface 18
!
interface 19
!
interface 20
!
interface 21
!
interface 22
!
interface 23
!
interface 24
!
interface 25
!
interface 26
!
!
!
!Sigh
This is just a switch config. This is not a script that configures the operating system. How the files from the SquashFS image make it to the root partition is still a mystery.
I wonder if there is a way to run code directly on this switch?
