pfSense on a WatchGuard Firebox x750e

<work in progress>

  1. Flash bios
  2. Flash pfSense
  3. 1st boot
  4. Turn down noise
  5. Optimization (boot config)
  6. LCD display
  7. Fix LEDS (NICs, Arm/Disarm)

Software used

  • Windows 7
  • Win32DiskImager
  • PuTTY
  • WinSCP

Flash bios

  • Flash FreeDOS to CF card. Add modified bios to flashed CF card.
  • Use console 9600 8N1
  • Boot into FreeDOS. 3 beeps means FreeDOS has started.
  • Follow bios upgrade procedure
    • backup
    • flash

To enable the Firebox to boot from a CF card larger then 512MB, we need to have bios access. You can try to use the internal VGA or PCIe connectors, but the ‘easiest’ way is to enable console redirect to access the bios via the serial port at the front of the Firebox.

You can only enable console redirect in the bios… So we have to flash the bios with a modified version that has console redirect enabled by default. Luckily, somebody already made a modified bios for this Firebox.

  1. Download <this> bootable FreeDOS image. It contains the awardflash program.

Flash pfSense

Now remove the CF card with FreeDOS from your Firebox and insert the CF card on which you just flashed pfSense.

1st boot

  • Use console 115200 8N1, Start up Firebox, use TAB key to enter bios
  • Set HD to CHS. Heads = 2, keep the rest as is (size is shown incorrect but doesn’t matter)
  • Use console 115200 8N1 to see pfSense booting
  • Follow initial setup

Change bios settings

Start a console 115200 8N1. Start up the Firebox and press TAB to enter the bios. Enter the menu Standard CMOS Features and press enter on IDE Channel 0 Master. First autodetect the new CF card. Then set IDE Channel 0 Master to Manual and Access Mode to CHS. Change Head to 2. Don’t change the other values. The capacity changes to a value below 512MB, but that’s ok. With these settings, the larger CF card will boot. Ppress ESC 2 times to go back to the previous menu and again 2 times for the main menu. Press Save & Exit Setup and confirm. The Firebox should reboot from the new CF card. You can leave the console at 115200 8N1 to watch pfSense boot.

Disable DMA

In the newer 2.2.x nano images, DMA access is no longer disabled by default. A fresh 2.2 install won’t boot from a CF card. The following has to be added to the /boot/loader.conf.local file: hint.ata.0.mode=PIO4.

If you add this line before an upgrade, it will be copied to the new 2.2 slice which will boot. We will consider a fresh install (the same applies if you already upgraded to 2.2 but your Firebox won’t boot). Interupt the boot loader when it is counting down from 4.

FreeBSD/x86 bootstrap loader, Revision 1.1
(root@pfs22-i386-builder, Mon Apr 13 20:28:31 CDT 2015)
Loading /boot/defaults/loader.conf
/boot/kernel/kernel text=0x11fb1a7 data=0x832e48+0x279e60 syms=[0x4+0xf3a10+0x4+0x16bd76]

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel] in 4 seconds...

Type '?' for a list of commands, 'help' for more detailed help.
OK

At the prompt enter:

set hint.ata.0.mode=PIO4
boot

pfSense now continues booting and enters the initial setup. For now we don’t set up VLAN’s and set WAN to sk0 and LAN to sk1 (sk0 is the left most port (port 0) and MSK3 is the right most port (port 7)). If everything went well, you will now see the console configuration menu.

First thing to do now is enabling secure shell (sshd) by choosing option 14. This way, you can use PuTTY or any other SSH client to connect to a terminal over the LAN.

Next thing is editing the /boot/loader.conf.local file to disable DMA after a reboot. This can be done with the webinterface on http://192.168.1.1 or the console shell (option 8) or WinSCP.

Webinterface default username/password: admin/pfsense

Choose menu Diagnostics > NanoBSD and click ‘Switch to Read/Write’. This way you can make changes to the file system.

Then go to Diagnostics > Edit file and browse to /boot and open the file loader.conf. Then change the filename to loader.conf.local and press save. Then delete the existing content of this file and add the rule

hint.ata.0.mode=PIO4

Click save and it’s done. Don’t forget to mount the filesystem back to read-only via Diagnostics > NanoBSD

If you use one of the other options, set the file permissions of loader.conf.local to 0644.

Turn down the noise

The Firebox makes a lot of noise with its three 10.000 RPM fans. If you don’t place the Firebox in a datacenter, you probably want to turn down the noise. Again, stephenw10 found a solution. He made a small program (WGXepc) which can adjust the fan speed (and a lot of other things).

You can use WinSCP the copy the program to the Firebox, or you can fetch is directly from the command line. Again you need to mount the filesystem read-write via the webinterface or by executing /etc/rc.conf_mount_rw. When your done, change it back to read-only with /etc/rc.conf_mount_ro

Put the file in /usr/local/bin with permissions set to 0755

Test the file by execution /usr/local/bin/WGXepc -l green

Steve Evans created a few scripts to automatically change the fan speed to control the the temperature. User wwwdrich made some modifications. Copy fanctrld.sh to /usr/local/etc/rc.d and copy fanctrld to /usr/local/sbin. Change the file permissions of both files to 0755.

Then edit /etc/rc.conf.local (or create this file if it does not exist already, permissions 0755) and add the following:

#!/bin/sh
fanctrld_enable="YES"        # Set to YES to enable the automatic fancontrol script

When you reboot the Firebox, the fanspeed will drop down to a comfortable level. When the temperature rises or an error occures, the script will automatically turn up the fans.

Optimization

Boot config

We already disabled DMA by adding the following to /boot/loader.conf.local

hint.ata.0.mode=PIO4

Then there could be problems with the MSK interfaces (the four most right interfaces). An interface becomes unresponsive and a watchdog timeout will be shown in the log. Add the following line to /boot/loader.conf.local to workaround this issue

hw.msk.msi_disable=1

sfgf

Arm/Disarm LED

The fancontrol script automatically sets the Arm/Disarm LED to green after booting. If the temperature is too high, this LED will flash green. If the temperature is way to high or an error occurs, the LED will flash red.

If you don’t use the fancontrol, you can use the command WGXepc -l green to turn the LED green. You can automate this proces by adding the package shellcmd to the webinterface and enter /usr/local/bin/WGXepc -l green as type shellcmd.

LCD Display

Add shellcmd package

Add LCDproc-dev package

http://forum.pfsense.org/index.php/topic,7920.msg344513.html#msg344513

NIC LEDs

The LEDS do not function as expected, they will only show activity, no link state. This can be changed by replacing the kernel drivers that set the LED control registers correctly.

pfSense 2.1.x is based on FreeBSD 8.3 and need other kernel modules then pfSense 2.2.x that is based on FreeBSD 10.1

Kernel modules FreeBSD 8.3 and instructions: https://doc.pfsense.org/index.php/PfSense_on_Watchguard_Firebox#Further_Enhancements_3

Kernel modules FreeBSD 10: https://forum.pfsense.org/index.php?topic=20095.msg460754#msg460754

Check if it works:

dmesg|grep LED (or cat /var/log/dmesg.boot | grep LED)

For the 8.3 modules you have to see ‘LED mod 1.3’ for the mskc0-3 interfaces and ‘LED mod 0.9’ for the skc0-3 interfaces.

For the 10 modules you will see ‘LED mod 2.2’ for all the interfaces.

Sources

https://doc.pfsense.org/index.php/PfSense_on_Watchguard_Firebox

http://forum.pfsense.org/index.php/topic,20095.0.html