As per my blog post on April 9 regarding the SQLSaturday website, the good news is that we’re back online. So far, feedback on the site has been overwhelmingly positive.
I first want to thank everyone for their patience last week: the organizers and sponsors who couldn’t access the site during the downtime, as well as the community members who have been waiting to hear what happened. As with any issue like this, our main priority was to rectify the situation. Therefore, we felt it best to wait until the site relaunched and all security vulnerabilities were fixed before sharing more specific details.
But of course, full transparency is important to us and to you. Now that we’re up and running again, here is the timeline of events that occurred over the past week:
- • On Monday, April 6, we were alerted to a potential security vulnerability that exposed the contact information (address, city, region, and twitter handle) of some sponsors. We immediately removed this information and decided to take down the entire sponsor page for further testing. The security of information regarding our community and sponsors is of the utmost importance to us, so we wanted to conduct a thorough review of the entire website, not just that specific issue.
- • By Monday night, we had decided to take the entire SQLSaturday site offline. We chose this option, rather than a rollback, because at the time, we estimated a rollback effort to be more time-consuming than simply taking the site offline and implementing the fix. In addition, we didn’t want to risk losing any new or changed data. We were able to minimize impact as best we could for the upcoming SQLSaturday events over the weekend of the 11th and 12th by providing access to the admin sites for the Huntington Beach and Madison SQLSaturday events.
- • The morning of Tuesday, April 7, we decided to ask community members for testing support. Our community comprises some of the best and brightest minds in the industry and it made sense to involve the users of the site in further testing.
- • The patch was completed by Tuesday night, making the site ready for testing by volunteers on Wednesday.
- • During the testing on Wednesday, April 8, a second potential vulnerability—an HTML injection vulnerability—was identified. Because of the seriousness of this potential issue, we decided Wednesday afternoon to keep the site offline for another day so that we could thoroughly research and correct the issue and complete final testing. As we began delving into the issue, we discovered that it also existed in the old site. So again, a rollback was not an option.
- • The problem was fixed late Wednesday night.
- • On Thursday, April 9, PASS IT and community-member testing was complete.
- • Satisfied with the security and usability of the site, we relaunched Thursday at 9:30pm EST.
PASS apologizes for this outage and for the difficulties it created for the SQLSaturday organizers, sponsors, speakers, and attendees. We thank those who provided feedback on the issues and the volunteers who stepped in to help test the solutions, particularly K. Brian Kelley (blog | @kbriankelley), Denny Cherry (blog | @mrdenny), and Argenis Fernandez (blog | @DBArgenis). To help prevent a similar issue in the future, we are looking at more extensive QA processes with a specific focus on ensuring site security. Although I believe we made the best possible decisions along this timeline, we will certainly take a different approach to future site revisions, including but not limited to earlier and wider security-based and functional testing by our volunteer experts and progressive change schedules.
Again, thank you for your patience. If you have any further feedback or questions, please email us at email@example.com.
PASS Board of Directors