The Outcasts were being attacked by dragons in "Alvin & the Outcasts".
As shown in "Heather Report, Parts 1 and 2", when the Outcast leader Alvin the Treacherous found out that dragons could be trained, he had his tribe capture as many dragons as they could hold in hope of using dragons for his own gain.
In "Defiant One", Hiccup, Toothless, and Snotlout got stranded on the island and got home before the Outcasts found them.
As shown in "We Are Family, Part 2", following the kidnapping of Hiccup, Alvin learned how to control dragons from the new dragon trainer of the Outcasts, Mildew, and earns the trust of a Whispering Death, bringing a new era to the Outcast Tribe.
As featured in "A View to a Skrill, Part 2", following the apparent "death" of Alvin, Dagur and his Berserker Tribe took control of the Outcasts and the Island.
During the events of "Cast Out, Parts 1 and 2, Dagur arranged a trap for Hiccup, which failed. Dagur then had tests done with dragons, observing their reactions to Dragon Root. He then held Stoick, later Hiccup and Toothless hostage at the Outcast Arena. However, Alvin and Mildew then freed the Whispering death and ended Dagur's control of Outcast Island.
As revealed in "Dragon Eye of the Beholder, Part 1", for 3 years, Dagur, and his loyal Berserkers and Outcast soldiers, were held here for their crimes. However, a traitor among the Outcasts gave Dagur a key, and he was able to escape.
During "A Time to Skrill", the Outcasts later abandoned the island when the Skrill came back and destroyed it, looking for Hiccup and Toothless. The dragon riders would later find what was left of the buildings and see the Outcasts where nowhere to be found, and then realize that the Skrill was already heading for Berk.
Shown in, "Last Auction Heroes", a group of dragon hunters then went to a tavern on Outcast island. It was there that Trader Johann stole a map from one of the hunters before being picked up by Hiccup and Toothless.