Horny Pony was released as the b-side of the single Gett Off, the first single from Prince’s thirteenth album Diamonds And Pearls and the first single to be credited to Prince and the New Power Generation. It was also included as the b-side of the following single, Cream. In 2001, it was included, with a computerized spoken intro (stating that the song is about dancing, and does not endorse extra-marital sex), as part of NPG Music Club Edition # 7.
A first version of this song was tracked on 7 September 1990 at Smile Garage Studio in Tokyo, Japan (on the same day as initial versions of Something Funky (This House Comes) and Call The Law). It was overdubbed at Paisley Park Studios, Chanhassen, Minnesota, a few days later, after Prince returned home from the Nude Tour. It was included as such as the ninth track on a December 1990, configuration of the Diamonds And Pearls album.
After rehearsing the song with the band, and having played it at the South American dates in January 1991, including at the Rock in Rio II Festival on 19 January 1991 (a.m.) that was broadcast on radio and television, Prince recorded his own stripped down version of it on 29 January 1991 at Paisley Park Studios, Chanhassen, Minnesota. This version, dubbed “Version 2”, was released in October 2023, as the sixth track of Vault Tracks part 2, one of the discs with previously unreleased tracks included with the Diamonds And Pearls Super Deluxe Edition' (and as the b-side of the Alice Through The Looking Glass single).
A third version was then recorded two days later on 31 January 1991 at Paisley Park Studios, Chanhassen, Minnesota with the band. This version was ultimately released as the b-side to Gett Off, and included as the seventh track on a 8 May 1991 configuration of the album. Also in May 1991, this version was included as the first track on an acetate of a Gett Off EP, which was aborted soon after compilation.
Horny Pony was replaced on the Diamonds And Pearls album by Gett Off in June 1991, but the track is still mentioned in the rap portion of Push (along with other songs on the album), and is printed on the corresponding tracklist on the packaging’s back cover (quoted from Push).