Back to Diane's HomePage
Kontakt

Diane's PS3 Hack Page




by

Diane Neisius aka ~medusa

Last update: April 26, 2010



Content

Important Note

Debian Linux on the PS3

Building and Testing the Xorhack Module

Examining LV1 Syscalls without exploit

Thoughts about Cell and PS3

Links

TO BE CONTINUED!




Important Note

If you're looking for warez or "isoldr" for you means "iso image loader for copied game DVDs" I recommend you follow this link.

*


This page is not for newbies or those faint ar heart. You should have at least an idea what Linux and it's kernel are good for, and you should not be afraid to use a text console on a computer. If a command like "find /usr -name libfoo* | nm -n | grep silly_symbol" reads like egyptean hieroglyphs for you, please get some Linux HOWTOs first. There are various at the web.
A Debian installation as well as installation of stuff like the exploit means doing things via keyboard commands. Sorry no fancy click-and-point front-end here. Once Debian is installed, you can of course use Gnome or any other desktop but for installation you will have to TYPE. Get used to it.
Also, you should have heard of the basics of C/C++ programming language. Linux means C, and that's it. A "C" HOWTO is not a bad idea if you're not familiar with it.

*


Wanna mail me? Decrypt this:
medusa ät diane minüs neisius döt de




Debian Linux on the PS3

A good HOWTO for Debian Linux on the PS3 is found in Geoff Levand's directory on kernel.org, and I'll keep close to that [1].

Basic Debian install
Installation of Debian on any machine these days is best done via net. Daily builds of actual bootloaders and installation kernels can be found at http://d-i.debian.org/daily-images/powerpc/daily/powerpc64/netboot. Get the newest petitboot "otheros.bld" from there (if not already done or you prefer kboot). Copy vmlinux, initrd.img and yaboot.conf into the root of an USB stick. Drop otheros.bld into a directory PS3/OTHEROS/ at the stick. Then use this USB stick for the usual way of installing "otheros" under PS3 GameOS. I suppose you've already set up a HDD partition for otheros.
Now boot into otheros. Debian installation kernel will come up if you choose "install" on the petitboot menu, and from this moment on everything will be the normal way of a Debian net installation. A few simple queries must be answered. Partitioning of the HDD can be done by accepting the installers default. Then go for a coffee as long as the installer downloads and unpacks the various software .deb archives.
PS3 Debian installation is just a bit tricky when it comes to end. You will be warned about no bootloader is installed. That's ok, we already have otheros.bld installed in the PS3's flash so press [enter]. The final message is "Installation completed" but with a warning that no appropriate kernel could be found. UNDER NO CIRCUMSTANCES PRESS ENTER HERE!!!
This must be done manually. Go to an alternate text console by pressing [ctrl]-[alt]-[F2]. Now you can press enter and will get a root shell. Enter "chroot /target". At this point you have access to the freshly installed Linux HDD root but still with running the installation kernel. Next is to download the new kernel image by "apt-get install linux-image-powerpc64". NOTE THAT THE PS3 DEBIAN INSTALL HOWTO HAS A TYPO HERE!!! You also can use aptitude to install but be aware you must install linux-image-powerpc64 for other packages may not fit to the kernel modules already installed!
After that, look up /boot which kernel version has been installed. Mine was vmlinux-2.6.32-3-powerpc64, which now must be set up in the boot config file.
To do so, use the nano editor to create the file /etc/kboot.conf and enter the following line:
debian=/boot/vmlinux-2.6.32-3-powerpc64 initrd=/boot/initrd.img-2.6.32-3-powerpc64 root=/dev/ps3da1
Of course, you will have to use your actual kernel image version you just looked up in /boot instead of 2.6.32-3. Now save the file, exit nano and by pressing [ctrl]-[alt]-[F1] you get back to the installer prompt which still waits for your acknowledge. If everything went well, you now safely can press enter and the freshly installed Debian will boot up after you removed the USB stick.
A sidenote: petitboot will show up all entries it can find in /etc/kboot.conf AND /etc/yaboot.conf in its menu. So it's a good idea to have kboot.conf as the main boot config which will be changed from time to time, and yaboot.conf with some emergency boot config as a backup which is never touched.

In my lab - My love would have kicked me off if I'd locked up our living room TV for days and weeks. So I decided to take the PS3 to my private laboratory (it's the same place where I do my laser and fusor experiments at other times; also, most of the work to get an oldie DEC PDP 11/03 back to life was done on this bench).
Hacks of a different kind, however... you see the Gnome desktop of a common Debian Linux installation run here on my shiny fat Black Lady.

Configuration yet to be done
Some stuff remains to be done. After booting in the new Debian on your PS3, switch to a text console ([ctrl]-[alt]-[F1]) and log in as root. Start aptitude and install p3-utils package. It is also a good idea to install the sources of the actual kernel you use. You will need them.
After this has been done, leave aptitude and edit /etc/rc.local . Before the line "exit 0" insert the two lines "mkswap /dev/ps3vram" and "swapon -p1 /dev/ps3vram". This will activate the vram swap next time you boot -- due to main RAM shortage on the PS3, nobody should waste 256k of graphics RAM. :)
Next thing is about the SPUs of the Cell processor. First, do a "mkdir /spu" then edit /etc/fstab with nano, append the line
none /spu spufs defaults 0 0
here and save.
One last thing remains to do should you want to use the hugepages which are enabled in the ppc64 kernel running. In this case, append one more line in /etc/fstab:
none /huge hugetlbfs defaults 0 0
and save and exit nano. Do a "mkdir /huge", edit once more /etc/rc.local and append "chmod 777 /huge" just after the swap commands. (This is neccessary because all directories in / will be reset to mode 755 during shutdown.)
Last thing is to set up the number of hugepages the kernel should reserve at boot. On a machine like the PS3 which is short in RAM it is a good idea to do this dynamically. Do by editing /etc/sysctl.conf (the pure Debian way is to create /etc/sysctl.d/local.conf instead), enter the line "vm.nr_overcommit_hugepages=4" here for dynamic reservation. Advantage is: dynamical reservation will not lock up precious RAM if hugepages are not used. Disadvantage: if the system ran for some time, page allocations will be shattered around entire RAM and maybe there is no gap of 16MB left free for hugepage allocation. So if you use hugepages on the PS3 with dynamic reservation, log in via text console just after bootup.
libhugetlbfs is not in the Debian repositories, you'll have to get it from sourceforge and build it for yourself. I tried 2.8 which worked without problems. The source tar.gz also contains a very good HOWTO where you can find more info.

Debian kexec install
Install kexec Debian package as usual (e.g., with aptitude). This is the 32-bit version, it will not load 64-bit kernel but execute it (don't ask me why, I just know it works).
Additionally, we need 64-bit kexec to load 64-bit kernel (it will crash on execute, just use it to load). So, install kexec again from source (the executable must be renamed to avoid conflicts):

(1) get & unpack kexec-tools-2.0.1-rc1.tar.gz
(2) ./configure --without-zlib CFLAGS=-m64 LD_FLAGS=-static
(3) make (will abort with error)
(4) at prompt, copy & paste last (failed) gcc command performed, DO NOT PRESS ENTER
(5) edit that line, insert -m64 -static after gcc and press enter, it now finishes build without any error
(6) cp build/sbin/kexec /usr/local/sbin/kexec64
and cp build/sbin/kdump /usr/local/sbin/kdump64

Use kexec/kexec64 with:
init 1
[you will be asked for the root password, type it]
/usr/local/sbin/kexec64 -l /boot/vmlinux-2.6.32-3-powerpc64 --initrd=/boot/initrd.img-2.6.32-3-powerpc64 --append="root=/dev/ps3da1 video=720p"
swapoff -a
sync
kexec -e

...which very well can be put into a shell script (see the warmboot script in the sputool tarball you can download here). This so far works for me only with the Debian ppc64 kernel shipped with the distribution and the 2.6.26 kernel rebuild from source with Debian 2.6.26 config file. Other ps3 related configs, default or some homebrewed I found in the web always hang when kexec -e was used.




Building and Testing the Xorhack Module

I'll give the good news first: xorhack will install and run without any problems under Debian if the right kernel is used. See below for more. :)
Originally using kernel 2.6.32 the exploit code for me didn't work just for try (without any hardware mod); it crashed with a segfault at 0xd000080080001000 (HTAB + 4k page). Looking up /proc/vmallocinfo, this area seems to be already allocated by the SPU init calls not the HTAB as supposed by the exploit code.
Seems there had been major changes in ioremap code lately. For kernel 2.6.26, ioremap stuff starts at 0xd000080080100000 which is a long way behind the HTAB area. More worse, also htab init has undergone a major revision in newer kernels. Up to now I couldn't find out where the htab now is located in VM. Also, for newer kernel 2.6.32 the xorhack module wants to use a symbol .irq_to_desc during build; however, the kernel doesn't export this so no chance to get a loadable module without a patch. Believe me, I had a few hard days until I found out what really was wrong. ;)
CONCLUSION: get the 2.6.26 kernel source for your distro and build & install it to work with the exploit. Kernel building isn't that hard, and there are a lot of HOWTOs around in the web, so I give no hints here. Just be aware if you boot a new Linux distro with a slightly outdated kernel it originally wasn't supposed to run a few failure messages during boot can result. Nothing really serious, but something to wonder about if you'll see it first time.

Running the exploit loop
This is best done in runlevel 1. The exploit uses critical timings so it's best as few other processes as possible run. If I run the exploit loop on the text console when the full desktop is up & running in the back, just the first "press button" message will be shown. The rest appears when the exploit loop already finished. Not very user-friendly. In runlevel 1 the messages show up one after the other just as they should. As I mentioned, currently I have not (yet) done the hardware mod so no button to press. But stay tuned. :)

Debian xorhack for newer kernels
Bad news is: xorhack won't build without errors for kernel 2.6.32 (well, warnings but these kick your module out when trying to insert it) if no minor patch is made. As far as my research is, the exploit code will not run anyway and also is the one making trouble during build (.irq_to_desc is undefined only here, use "nm exploit.o" to check). The simplest solution is to exclude it from module build (edit the makefile in kmod directory and delete "exploit.o" in the line starting with "obj-m=..."). Further this requires editing of xorhack_kmod.c to uncomment the call to ps3_exploit() which no longer is available. You will not have the exploit but at least the xorhack syscall interface for comfortable experiments about lv1 syscalls. Most of the time I developed sputool (download here) I worked with this configuration. Maybe it's "xorhack lite" but it's at least something if you don't want to use other kernel than you have.
The more tricky way is about a patch of your current kernel. However, you'll have to compile kernel from sources anyway if you want to build a module. So it's a minor effort to do a simple patch before. To allow for proper xorhack build a kernel must export the symbol .irq_to_desc. To enable, edit the file [kernel-src]/kernel/irq/handle.c by the editor of your choice.
Look for a short function definition like this:
struct irq_desc *irq_to_desc(unsigned int)
{
	[...some code...]
}
It should appear twice, at line 192 and 237 for we have an #ifdef...#else...#endif preprocessor structure in the file. Append in both cases after the closing "}" bracket the line
EXPORT_SYMBOL(irq_to_desc);
and save the file. Then compile and install your kernel, and after that xorhack.ko should build without a warning.
As mentioned above, the exploit code will not run but at least again you can use the xorhack lv1 syscall interface for experimentation. You may also reproduce the segfault at 0xd000080080001000; it would be important to find out where the htab is mapped at newer kernels so we get a chance to do the exploit also for them.




Examining LV1 Syscalls without exploit

My first intention was to unlock the 7th SPU by simple lv1 manipulation. Geohot mentioned the decryptor SPU is idle under Linux, so I digged deep into the SPU related kernel source. However, everything inside the PME is locked quite hard so if you don't want to open up your Black Lady (which I do not [yet]) that dream passes away...
I just set things up to be ready for exploit. It turned out this must be done with older kernel under Debian. Xorloser mentioned he used Ubuntu 8.10 which had kernels 2.6.25 and 2.6.27; the closest Debian kernel is 2.6.26, and indeed for the latter the exploit code at least runs.
What is left:
Sputool modification of the current partition repository already works if there are only the repo SPU values are decreased (sputool -d) and the the Debian ppc64 kernel is restarted using kexec. Linux then comes up with the number of SPUs you set up but that's fewer than before so it's boring. It just demonstrates that even without exploiting the PS3 things can be manipulated without letting the HV know about (I feel this is the only one important result of the experiments I did). Repo entries can be queried, modified, created and deleted. See below or sputool.c for examples.
However, increasing SPU number is not that simple, it will crash the kernel during kexec if just the repo values were increased before. Creating a SPU does not better, and it cannot be enabled. Destroying SPUs makes things more worse, and don't even think to kexec after you used sputool -x.

So I think without exploit I came to an end here. Shooting down the decrytor SPU of the system, manipulation of the sys.be repo entries and restarting the 7th SPU in user mode is left to exploited PS3s - if at all.
Nevertheless, thanx to geohot and xorloser for making even the bits I've done possible.

LV1 calls I experimented with

lv1 call No.lv1 function name arguments in/outdescription
90lv1_create_repository_node(key1,key2,key3,key4,v1,v2)6 / 0create in current partition repo node with key1-key4, init with v1 and v2 values
91lv1_get_repository_node_value(p_id,key1,key2,key3,key4,&v1,&v2)5 / 2read values of repo node key1-key4 of partition p_id into v1 and v2
92lv1_modify_repository_node_value(key1,key2,key3,key4,v1,v2)6 / 0modify in current partition repo node key1-key4 with values v1, v2
93lv1_remove_repository_node(key1,key2,key3,key4)4 / 0remove in current partition repo node key1-key4


Of the lv1 calls I played around with, No. 91 already has been documented (see, for example, wiki.ps2dev.org). That's the get repo value call which is used often in the SPU related kernel code.
However, repo nodes also can be created, modified and deleted - at least in the logical partition of the kernel (other partition repos are read-only). Note all the examples below already work without the exploit! For example, if you do the following:

#define PS3_LPAR_ID_LINUX	        0x0000000000000002
#define BI				0x0000000062690000
#define SPUN				0x7370756e00000000
#define SPURSVN				0x7370757273766e00
#define SPURSV				0x7370757273760000
#define PS3_SPU_RESOURCE_TYPE_EXCLUSIVE 0x8000000000000000

[...]

main()
{
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPUN, 0, 0, &n_spus, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPURSVN, 0, 0, &n_res_spus, &v2);

   status = lv1_modify_repository_node_value( BI, SPUN, 0, 0, n_spus+1, 0);
   status = lv1_modify_repository_node_value( BI, SPURSVN, 0, 0, n_res_spus+1, 0);
   status = lv1_create_repository_node( BI, SPURSV, n_res_spus, 0, 
				        PS3_SPU_RESOURCE_TYPE_EXCLUSIVE, n_res_spus);
}


You will read out bi.spun and bi.spursvn values, modify both by incrementing them by one and create a new node bi.spursv.#(n_res_spu). Same also works in the opposite direction:

#define [see above]

[...]

main()
{
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPUN, 0, 0, &n_spus, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_LINUX, BI, SPURSVN, 0, 0, &n_res_spus, &v2);

   status = lv1_modify_repository_node_value( BI, SPUN, 0, 0, n_spus-1, 0);
   status = lv1_modify_repository_node_value( BI, SPURSVN, 0, 0, n_res_spus-1, 0);
   status = lv1_remove_repository_node( BI, SPURSV, n_res_spus-1, 0 ); 
}

Here bi.spun and bi.spursvn are read and decremented by one; the now obsolete node bi.spursv.#(n_res_spu-1) is deleted.
Repo values also can be read from other partitions as is the protected memory environment (PME) where information about the SPU setup can be found:

#define PS3_LPAR_ID_PME			0x0000000000000001
#define SYS				0x0000000073797300
#define BE				0x6265000000000000
#define SPURSVSL			0x737075727376736c
#define AUSRSPUN			0x617573727370756e
#define ASYSSPUN			0x617379737370756e	

[...]

main()
{
   status = lv1_get_repository_node_value( PS3_LPAR_ID_PME, SYS, BE, 0, SPURSVSL, &v1, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_PME, SYS, BE, 0, AUSRSPUN, &v1, &v2);
   status = lv1_get_repository_node_value( PS3_LPAR_ID_PME, SYS, BE, 0, ASYSSPUN, &v1, &v2);
}

You will find sys.be.0.ausrspun = 6, sys.be.0.asysspun = 1 (obviously the number of SPUs for user and system) and sys.be.0.spursvsl = 0xc0c0c0c0c0c0a100. The latter reads maybe "SPU reservation security level (?)" and has six times the byte 0xC0 (for the 6 user SPUs?), one byte 0xA1 (for the system SPU?) and one byte 00 (for the braindead SPU?) in a doubleword. In all 3 cases is v2 == 0. Interesting stuff to play with, but it's in HV r/o protected memory and currently I do not (yet) have the exploit hardware to proceed. Stay tuned.

P.S.: the examples above are slightly simplifed C source out of the sputool source. If you like to play with it, download the tar.gz from my ../download/ps3 directory.




Thoughts about Cell and PS3

I got my PS3 for the explicit purpose of running Linux on it. I wanted to program the Cell, and still the PS3 is the only Cell machine I can pay for (probably like most of us). Yes, I also like to play from time to time; my PS3 favourite is fl0wers which I downloaded to the GameOs partition. When it comes to shooters, I don't like games which require hours of manual study and weeks of pratical excercise about gamepad handling. Shooters had no new ideas since the 1980ies, and if I'm in a "nuke 'em" mood, I throw a "Space Invaders" microdrive cartridge in my 25-year old Sinclair QL. :)
About PS3 Linux, I was happy the way it was. Played a while around with YDL 6, but for serious work I installed my preferred distro Debian on it. Often heard flames about the missing RSX support, but hey boyz, it does framebuffer graphics. When I did my first Linux install long long time ago it was a great luck if you've got a framebuffer X server running. Graphics drivers for Linux were very very rare those days. On the PS3, the only thing I missed was the 7th SPU. I paid for 7, I want 7. That's the reason why I was really interested in the exploit.
Of course I do not favour Sony's decision to remove Linux support. Ok, put your PSN in your..., I very well can live with FW 3.15, and I now let my Black Lady never again connect to the web when GameOs is running. Sorry, Stanford, no more folding@home donations from me. And should ever come a PS4, I certainly will never buy one without Linux support.

Little Nerdie - To me, the PS3 has supercomputer capabilities in more than one aspect. The most important of course is its Cell processor. However, even its design reminds a bit to the old mainframes - as is demonstrated here by a Momoko doll out of my love's collection. I have a true weakness for puppet-sized mainframes. In the back you see a case-modded P4 in the style of 1980ies IBM and Control Data mainframes I designed in 2004. Oh, those dark acrylic front covers!!!
In this context, the PS3 has a faint smell of Cray 2 or something like that famous oldie. Too bad today's mainframes are just lousy 19" racks in a row. :(


Not only for its supercomputer capabilities I like the Cell. In my opinion, it is endpoint of a long line of processor architecture inhertage which is: PDP11 > 68000 > PowerPC > Cell. I always favoured this line more than the line of x86 architectures (yes I also liked 6502 more than Z80 in the 1980ies). Love to hack their assemblers.
Of course the theoretical 200 Gflop/s of the Cell (150 Gflop/s for PS3 with only 6 SPUs) cannot be reached for realistic user code. The peak performance I got was 28.8 Gflop/s during the fftw library benchmark which does fast Fourier transform. Goddess, the NEC SX4 mainframe had that much which was worlds fastest computer on the TOP 500 list just 10 years ago!!!
This also holds for the hypervisor. I came in touch first with such when using VM/CMS on an IBM 3900 on few occasions (1991, if I remember right). Hypervisor is mainframe stuff, and Cell and HV belong to each other. The Cell even has HV privileged instructions, besides superuser instructions and common user instructions. Sony was often blamed to set up the HV just for copy protection, but as far as I know the business (I work as algorithm developer in the Advanced Engineering department of a big car electronics manufacturer) Sony simply licensed the IBM reference hypervisor and stripped it down for their purposes. If you lookup chapter 11.8 of the Cell/BE programming handbook [3], you find an example HV syscall which looks EXACTLY like a lv1 call.
As of these days more is known about the PS3 boot process, it also points to that direction. Lv0 is the bootloader, which sets up the decryptor in an isolated SPU; next lv1 is started which is the hypervisor. Now the kernel of choice is started, may it be Linux or GameOS/lv2. Both of them run under the same hypervisor environment (noobs often stress in countless forum posts lv1 is the Linux hypervisor and lv2 the GameOS hypervisor; that's simply not true. Lv2 is the GameOs kernel which runs under lv1 as well as Linux does). That's also the reason why just 6 SPUs are available under Linux: when lv1 starts it yet has no idea if Linux or GameOS will be loaded later so the decryptor SPU remains isolated. By theory it should even be possible to run both in parallel under lv1. Maybe RAM would be a bit short, but exactly this is the original purpose of a hypervisor.

Thanks to Kuratagi-san for the design of such a wonderful machine.


TO BE CONTINUED!!!





Links


Debian homepage -- Everything the worshipper of Pure GNU/Linux needs :)

ps2dev.org wiki page -- collection of information about PS3 lv1 calls, among lots of other info

IBM developer works -- download of Cell handbooks

Xorloser's blog -- download of the xorhack 2.0 source tarball

../download/ps3 -- my ps3 download directory



*

References

[1] PS3 Debian Install HOWTO, http://www.kernel.org/pub/linux/kernel/people/geoff/cell

[2] Scarpino, M.: Programming the Cell processor. Pearson Education Inc, Boston, 2009. 

[3] Cell Broadband Engine Programming Handbook 1.12, IBM, Sony, Toshiba 2009.

[4] PowerPC Microprocessor Family: the Programming Environments, IBM, Motorola 1997.

[5] Synergistic Processor Unit Instruction Set Architecture 1.2, IBM, Sony, Toshiba 2007.




Created: Apr. 26, 2010
© 2010 Diane Neisius