Diane's PS3 Hack Page

In my lab - My love would have kicked me off if I'd locked up our living room TV for days and weeks. So I decided to take the PS3 to my private laboratory (it's the same place where I do my laser and fusor experiments at other times; also, most of the work to get an oldie DEC PDP 11/03 back to life was done on this bench). Hacks of a different kind, however... you see the Gnome desktop of a common Debian Linux installation run here on my shiny fat Black Lady.

Configuration yet to be done
Some stuff remains to be done. After booting in the new Debian on your PS3, switch to a text console ([ctrl]-[alt]-[F1]) and log in as root. Start aptitude and install p3-utils package. It is also a good idea to install the sources of the actual kernel you use. You will need them.
After this has been done, leave aptitude and edit /etc/rc.local . Before the line "exit 0" insert the two lines "mkswap /dev/ps3vram" and "swapon -p1 /dev/ps3vram". This will activate the vram swap next time you boot -- due to main RAM shortage on the PS3, nobody should waste 256k of graphics RAM. :)
Next thing is about the SPUs of the Cell processor. First, do a "mkdir /spu" then edit /etc/fstab with nano, append the line
none /spu spufs defaults 0 0
here and save.
One last thing remains to do should you want to use the hugepages which are enabled in the ppc64 kernel running. In this case, append one more line in /etc/fstab:
none /huge hugetlbfs defaults 0 0
and save and exit nano. Do a "mkdir /huge", edit once more /etc/rc.local and append "chmod 777 /huge" just after the swap commands. (This is neccessary because all directories in / will be reset to mode 755 during shutdown.)
Last thing is to set up the number of hugepages the kernel should reserve at boot. On a machine like the PS3 which is short in RAM it is a good idea to do this dynamically. Do by editing /etc/sysctl.conf (the pure Debian way is to create /etc/sysctl.d/local.conf instead), enter the line "vm.nr_overcommit_hugepages=4" here for dynamic reservation. Advantage is: dynamical reservation will not lock up precious RAM if hugepages are not used. Disadvantage: if the system ran for some time, page allocations will be shattered around entire RAM and maybe there is no gap of 16MB left free for hugepage allocation. So if you use hugepages on the PS3 with dynamic reservation, log in via text console just after bootup.
libhugetlbfs is not in the Debian repositories, you'll have to get it from sourceforge and build it for yourself. I tried 2.8 which worked without problems. The source tar.gz also contains a very good HOWTO where you can find more info.

Debian kexec install
Install kexec Debian package as usual (e.g., with aptitude). This is the 32-bit version, it will not load 64-bit kernel but execute it (don't ask me why, I just know it works).
Additionally, we need 64-bit kexec to load 64-bit kernel (it will crash on execute, just use it to load). So, install kexec again from source (the executable must be renamed to avoid conflicts):

(1) get & unpack kexec-tools-2.0.1-rc1.tar.gz
(2) ./configure --without-zlib CFLAGS=-m64 LD_FLAGS=-static
(3) make (will abort with error)
(4) at prompt, copy & paste last (failed) gcc command performed, DO NOT PRESS ENTER
(5) edit that line, insert -m64 -static after gcc and press enter, it now finishes build without any error
(6) cp build/sbin/kexec /usr/local/sbin/kexec64
and cp build/sbin/kdump /usr/local/sbin/kdump64

Use kexec/kexec64 with:
init 1
[you will be asked for the root password, type it]
/usr/local/sbin/kexec64 -l /boot/vmlinux-2.6.32-3-powerpc64 --initrd=/boot/initrd.img-2.6.32-3-powerpc64 --append="root=/dev/ps3da1 video=720p"
swapoff -a
sync
kexec -e

...which very well can be put into a shell script (see the warmboot script in the sputool tarball you can download here). This so far works for me only with the Debian ppc64 kernel shipped with the distribution and the 2.6.26 kernel rebuild from source with Debian 2.6.26 config file. Other ps3 related configs, default or some homebrewed I found in the web always hang when kexec -e was used.

struct irq_desc *irq_to_desc(unsigned int)
{
	[...some code...]
}

#define PS3_LPAR_ID_LINUX	        0x0000000000000002
#define BI				0x0000000062690000
#define SPUN				0x7370756e00000000
#define SPURSVN				0x7370757273766e00
#define SPURSV				0x7370757273760000
#define PS3_SPU_RESOURCE_TYPE_EXCLUSIVE 0x8000000000000000

[...]

main()
{
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPUN, 0, 0, &n_spus, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPURSVN, 0, 0, &n_res_spus, &v2);

   status = lv1_modify_repository_node_value( BI, SPUN, 0, 0, n_spus+1, 0);
   status = lv1_modify_repository_node_value( BI, SPURSVN, 0, 0, n_res_spus+1, 0);
   status = lv1_create_repository_node( BI, SPURSV, n_res_spus, 0, 
				        PS3_SPU_RESOURCE_TYPE_EXCLUSIVE, n_res_spus);
}

#define [see above]

[...]

main()
{
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPUN, 0, 0, &n_spus, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPURSVN, 0, 0, &n_res_spus, &v2);

   status = lv1_modify_repository_node_value( BI, SPUN, 0, 0, n_spus-1, 0);
   status = lv1_modify_repository_node_value( BI, SPURSVN, 0, 0, n_res_spus-1, 0);
   status = lv1_remove_repository_node( BI, SPURSV, n_res_spus-1, 0 ); 
}

#define PS3_LPAR_ID_PME			0x0000000000000001
#define SYS				0x0000000073797300
#define BE				0x6265000000000000
#define SPURSVSL			0x737075727376736c
#define AUSRSPUN			0x617573727370756e
#define ASYSSPUN			0x617379737370756e	

[...]

main()
{
   status = lv1_get_repository_node_value( PS3_LPAR_ID_PME, SYS, BE, 0, SPURSVSL, &v1, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_PME, SYS, BE, 0, AUSRSPUN, &v1, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_PME, SYS, BE, 0, ASYSSPUN, &v1, &v2);
}

Little Nerdie - To me, the PS3 has supercomputer capabilities in more than one aspect. The most important of course is its Cell processor. However, even its design reminds a bit to the old mainframes - as is demonstrated here by a Momoko doll out of my love's collection. I have a true weakness for puppet-sized mainframes. In the back you see a case-modded P4 in the style of 1980ies IBM and Control Data mainframes I designed in 2004. Oh, those dark acrylic front covers!!! In this context, the PS3 has a faint smell of Cray 2 or something like that famous oldie. Too bad today's mainframes are just lousy 19" racks in a row. :(

TO BE CONTINUED!!!

[1] PS3 Debian Install HOWTO, http://www.kernel.org/pub/linux/kernel/people/geoff/cell

[2] Scarpino, M.: Programming the Cell processor. Pearson Education Inc, Boston, 2009. 

[3] Cell Broadband Engine Programming Handbook 1.12, IBM, Sony, Toshiba 2009.

[4] PowerPC Microprocessor Family: the Programming Environments, IBM, Motorola 1997.

[5] Synergistic Processor Unit Instruction Set Architecture 1.2, IBM, Sony, Toshiba 2007.

• Back to Diane's HomePage