If the integration of SharePoint 2013 and TFS 2012 goes poorly, it may be necessary to uninstall/reconfigure your SharePoint Extensions. The location of this option however, is not easy to locate.
You may end up with a screen that looks something similar to the above.
After an hour of searching, I finally located the option to remove the installed SharePoint Extension.
Problem solved – you can then reinstall your SharePoint Extensions as normal.