Since version 5.00 of the PSP firmware came out in 2008, the handheld has had a direct link to the PSN store, and according to my Download List, I've been downloading stuff since November 2007 (presumably via PC, when that function moved PSN away from being PS3-only).
The first game I ever downloaded was the WarHammer 40K Squad Command demo, followed by a bunch of the early PSOne games; Cool Boarders, Crash Team Racing, MediEvil and WipEout. Then came actual PSP games with Beats being the first of the original titles, then Flow and SuperStardust Portable.
After that there was a bit of a pause with the odd burst of Patapon, before things got really interesting with the release of Final Fantasy VII and Vagrant Story, Ridge Racer Type 4, but since then, most of my purchase history has been a bunch of great minis and the odd on-sale PSP title.
As the PSP element of the PSN winds down, helped only by the odd PSOne classic, the sterling efforts of Ghostlight and a few others, I hope Sony Europe (and probably Sony America) remembers to take stock of just how poor its efforts have been compared to Japan's and do a whole let better for the Vita's edition of the store.
What was your favourite PSN memory for PSP? I remember getting Manhunt for free due to a PSN screw-up and was probably most surprised by the simple pleasure of Half Minute Hero than any other downloaded title, something I'd never have picked up in the shops.