Probably the most challenging step of getting started with ps2dev is getting the environment right. You will need the ps2 setup with a method of loading your programs aswell as compilers and tools to develop your programs.
To load your programs there are a number of different loaders that you can use. Check out the loaders section on this site for downloads.
The most common method of loading programs that you write or written by others is using ps2link. ps2link requires that you own a ps2 ethernet adapter. The pc and ps2 are connected via ethernet using a cross over cable or hub. ps2link is loaded on the PC. A client such as inLink is used on the PC to send programs to the PS2.
Another method of loading and testing the programs you write onto your PS2 is via Naplink. Naplink requires a PL-2301 USB<->USB cable which you plug into your PC and PS2. You start Naplink on your PC then load up your PS2 with the Naplink server.
To load ps2link or naplink on your PS2 you either need to have a mod-chip fitted or use the ps2-independence exploit.
Naplink Demo Disk Swap
Getting Started with Naplink/PS2(Messiah mod-chip)
3stars.elf built on gnu NT.