Apple has updated its iTunes software to correct a security shortcoming that offered the potential for miscreants to mount man-in-the-middle attacks and appears to have played a central role in the infamous Ghost Click botnet scam.

iTunes 10.5.1, released on Monday, is a cross-platform update that addresses a flaw that’s most acute on Windows systems. Prior to the update, hackers had the potential to intercept update queries between the iTunes client on a user’s Windows machine and Apple. This might have been abused to offer a Trojaned version of iTunes, or – more likely and much easier to pull off – redirect surfers to a site punting fake anti-virus (AKA scareware) or running click-fraud scams.

The threat is most acute when Apple Software Update for Windows is not installed. In these cases, a user’s default browser might be opened to a location under the control of hackers that poses as an Apple site.

Apple Software Update is included with OS X so the risk is a fair bit less for Mac fans. Nonetheless Mac users also need to update.

Fixing the flaw on both Mac and Windows machines involves enforcing updates via a secure connection, something that probably ought to have been applied as general good practice in the first place. Apple’s advisory makes no mention if attacks based on the vulnerability had actually taken place but we’re pretty sure this relates to the DNS Changer scam, which caused machines running Apple’s Mac OS X, as well Windows PCs, to rely on rogue domain name system servers set up by hackers.

The malware was used to establish a 4 million drone botnet, used to hijack browsing sessions on infected machines. Fraudulent web pages appeared when victims attempted to visit Netflix, the US Internal Revenue Service, Apple’s iTunes and other services. Six Estonian suspects have been charged, and one Russian suspect remains at large, following the high-profile Ghost Click takedown operation earlier this month.

An FBI press release on the Ghost Click takedown specifically cites iTunes as an example of how the alleged fraud operated. The mechanism of the attack is uncannily similar to that addressed by the iTunes update.

DNSChanger was used to redirect unsuspecting users to rogue servers controlled by the cyber thieves, allowing them to manipulate users’ web activity. When users of infected computers clicked on the link for the official website of iTunes, for example, they were instead taken to a website for a business unaffiliated with Apple Inc that purported to sell Apple software. Not only did the cyber thieves make money from these schemes, they deprived legitimate website operators and advertisers of substantial revenue.

Trend Micro, one of several security firms and academics that assisted police in the takedown, has a good write-up on how the Ghost Click scam operated from a technical perspective here.

The scam netted the alleged cybercrooks an estimated $14m since 2007 via clickfraud, fake anti-virus and malvertising rogue (unlicensed) pharmacy websites.

Normally we’d like to phone Apple up and put our theory to it that the iTunes flaw played a central role in the Ghost Click fraud over several years. Unfortunately, experience tells us that Apple, unlike other vendors, never responds to such calls. This is a shame because there are still wider lessons to be drawn from the Ghost click takedown that could help the industry as a whole improve security practices and help prevent future repetitions of this type of scam. ®