Status: The Present

Statuses current as of 0 seconds ago.
Next refresh in 0 seconds.
Server time on pageload: 2012-05-16 15:31:32 -0700
 

Status: The Future

There are currently no scheduled maintenance windows.

Status: The Past

 

Saturday, May 5th #

2012-05-05 11:00pm12:00am -07:00 (PST) 2012-05-06 06:002012-05-06 07:00 +00:00 (GMT)

Database maintenance, rescheduled

Impacted Systems: app, admin

We have rescheduled our routine maintenance to optimize the large database table on our primary database server.

Once again, we’ll need to shut off access to all FoxyCart Stores for a few minutes. Our DNS failover system will handle traffic during the downtime, however, your FoxyCart store and the FoxyCart Admin may be unavailable for up to ten (10) minutes.

We’ll do our best to minimize the downtime. Thank you for your patience!

 

Thursday, May 3rd #

2012-05-03 10:10pm11:10pm -07:00 (PST) 2012-05-04 05:102012-05-04 06:10 +00:00 (GMT)

www.foxycart.com downtime

Impacted Systems: www

The host for www.foxycart.com went down temporarily and DNS failed over. The issue has been resolved but DNS may take a little while to update back. FoxyCart Stores were not impacted.

Update May 4, 2012 09:30 PST: HiFi, our web site’s host reports: “Our datacenter, Softlayer had a ~10 minute network issue that affected some parts of the country, HiFi itself was uninterrupted.”

 

Saturday, Apr 28th #

2012-04-28 11:00pm12:00am -07:00 (PST) 2012-04-29 06:002012-04-29 07:00 +00:00 (GMT)

Database maintenance

Impacted Systems: app, admin

We need to perform routine maintenance and optimize a large database table on our primary database server. In order to do this safely we’ll need to shut off access to all FoxyCart Stores for a few minutes. Our DNS failover system will handle traffic during the downtime, however, your FoxyCart store and the FoxyCart Admin may be unavailable for up to ten (10) minutes.

We’ll do our best to minimize the downtime. Thank you for your patience!

Update, 2012 Apr 29 0:00 PST: Due to a scheduling conflict we have postponed this maintenance window and will reschedule it soon.

 

Monday, Apr 9th #

2012-04-09 9:00pm11:00pm -07:00 (PST) 2012-04-10 04:002012-04-10 06:00 +00:00 (GMT)

Maintenance on AUX

Impacted Systems: forum, wiki, affiliate

We’re taking our AUX server down for a reboot tonight. AUX hosts forum and wiki so they may be unavailable for a brief time during the maintenance window. FoxyCart Stores will NOT be affected.

Update April 10, 2012 21:31 PST: Our maintenance has been completed.

 

Tuesday, Mar 27th #

2012-03-27 9:00pm10:00pm -07:00 (PST) 2012-03-28 04:002012-03-28 05:00 +00:00 (GMT)

Planned Maintenance: configuration changes

Impacted Systems: app, admin

As a necessary step in releasing our new API, we are making some planned configuration changes on our primary app server. Downtime is not expected for this maintenance window, however, your FoxyCart Store may be briefly affected. Stay tuned, and thank you!

Update March 28, 2012 21:10 PST: Our maintenance window has been completed. Short and sweet!

 

Monday, Mar 26th #

2012-03-26 5:57pm6:04pm -07:00 (PST) 2012-03-27 00:572012-03-27 01:04 +00:00 (GMT)

Unexpected Network Issues [Pending]

Impacted Systems: app, admin

At 5:57pm PST there was an issue impacting connectivity. We are currently looking into the cause.

UPDATE 6:04pm PST: Connectivity has been restored. We will update when we have more info.

UPDATE 6:16pm PST: Our hosting provider has explained the problem. Though we do have a backup application environment at a separate datacenter, it isn’t yet ready (though is close) to seamlessly handle an outage like this. We apologize for the service interruption, and are working towards getting our hot failover environment ready.

 

Saturday, Mar 24th #

2012-03-24 2:00am4:00am -07:00 (PST) 2012-03-24 09:002012-03-24 11:00 +00:00 (GMT)

Network issues at primary datacenter

Impacted Systems: app, admin

Our hosting company is having an unexpected network issue and FoxyCart stores are currently unavailable. We are working with them to find a solution ASAP and will update shortly.

Update March 24, 2012 02:35 PST: Our systems are back up. Working with our host to get to the root cause of the downtime.

Update March 24, 2012 02:58 PST: Our hosting company has confirmed that their systems are back to normal and are investigating the scope and root cause. All FoxyCart systems normal.

 

Wednesday, Mar 7th #

2012-03-07 9:40am10:40am -07:00 (PST) 2012-03-07 16:402012-03-07 17:40 +00:00 (GMT)

Service interuption on www

Impacted Systems: www, forum, wiki, affiliate

While doing some routine maintenance we ran into an unexpected problem. The site is back up now. FoxyCart Stores were not affected.

 

Monday, Feb 20th #

2012-02-20 5:00am9:00am -07:00 (PST) 2012-02-20 12:002012-02-20 16:00 +00:00 (GMT)

Network connectivity on www server

Impacted Systems: www, forum, wiki

Our www server is currently unreachable. We’re investigating. FoxyCart Stores are working as normal.

Update February 20, 2012 05:17 PST: Slicehost reports that a DDoS attack on one of their customers is causing network troubles on their end. Their network team is working on it.

Update February 20, 2012 05:51 PST: Slicehost reports that the network issues are beginning to resolve.

Update February 20, 2012 08:48 PST: Systems have been normal for the past few hours. This issue is now completely resolved.

 

Tuesday, Feb 14th #

2012-02-14 4:20am4:30am -07:00 (PST) 2012-02-14 11:202012-02-14 11:30 +00:00 (GMT)

Service Interruption on app1

Impacted Systems: app, admin

Our application server became unavailable for about two minutes this morning but is now back up. We are working with our hosting company to track down and resolve this issue.

Update February 14, 2012 06:48 PST: Here’s the update from our hosting company: “At approximately 4:25 PST our network engineers were notified of a brief connectivity slowdown from emanating from one of our upstream providers. After investigating the issue was resolved by our network engineers and all has returned to normal.”

All systems are normal at this time.

 

Monday, Feb 13th #

2012-02-13 9:00pm10:00pm -07:00 (PST) 2012-02-14 04:002012-02-14 05:00 +00:00 (GMT)

Minor maintenance on app1

Impacted Systems: app, admin

We are performing minor routine maintenance on our primary application server during this window. Your FoxyCart store will be available throughout the maintenance window.

Update February 14, 2012 21:10 PST: Our maintenance has been successfully completed. All systems normal.

 

Monday, Feb 6th #

2012-02-06 11:10am12:00pm -07:00 (PST) 2012-02-06 18:102012-02-06 19:00 +00:00 (GMT)

DNS Failover on www

Impacted Systems: www, forum, wiki, affiliate

Due to an invalid setting, our DNS monitoring service triggered a false positive and failed over our www, forum, and wiki for a few minutes. We’ve corrected the problem and those sites will be back shortly. FoxyCart Stores were not affected.

 

Tuesday, Jan 24th #

2012-01-24 6:00am10:00am -07:00 (PST) 2012-01-24 13:002012-01-24 17:00 +00:00 (GMT)

v0.6.0 Subscriptions Processing Delay

Impacted Systems: app

There was a bug this morning that prevented subscriptions for stores on FoxyCart v0.6.0 from processing as normal. We have identified the issue and are working on a fix, both to run the subscriptions today as normal (albeit some hours delayed) and to prevent this from happening in the future. We’ll update here when that fix is made and the subscriptions are processed.

Update 10am PST: Resolved. All subscriptions have been processed as normal.

 

Monday, Jan 23rd #

2012-01-23 12:00am1:35pm -07:00 (PST) 2012-01-23 07:002012-01-23 20:35 +00:00 (GMT)

USPS First Class rates not being returned

Impacted Systems: app

We are currently working on an issue that’s preventing USPS First Class rates from being returned. This is our top priority until it is resolved. We will be providing updates on our forum here and here, and will close this status alert when it’s fully resolved.

UPDATE 13:35 PST: This has been fully resolved. Please see the above linked forum threads for details.

 

Saturday, Jan 14th #

2012-01-14 2:00am2:30am -07:00 (PST) 2012-01-14 09:002012-01-14 09:30 +00:00 (GMT)

Maintenance window on app1

Impacted Systems: app, admin

We are performing scheduled maintenance on our primary application server to upgrade some operating system packages. FoxyCart Stores may be offline for a minute or two during this window.

Update January 14, 2012 02:17 PST: Maintenance window completed. All systems normal.

 

Wednesday, Jan 11th #

2012-01-11 12:00pm4:00pm -07:00 (PST) 2012-01-11 19:002012-01-12 23:00 +00:00 (GMT)

Unusually High Load on our Primary APP Server

Impacted Systems: app, admin

We’re currently experiencing high loads, resulting in slow pageloads. We’re working with our hosting provider to determine if this is a localized issue and will update this status once we know more.

Update January 11, 2012 14:08 PST: We have the loads under control and are continuing to monitor the situation. We found a memory leak in our web server processes and were able to resolve it by forcibly restarting them.

Update January 11, 2012 15:50 PST: Loads continue to fluctuate. We are scheduling an emergency maintenance window for this evening to take the server offline and reboot it.

 

Wednesday, Jan 11th #

2012-01-11 9:00pm10:00pm -07:00 (PST) 2012-01-12 04:002012-01-12 05:00 +00:00 (GMT)

Emergency Maintenance: app1

Impacted Systems: app

As part of diagnosing and resolving the high load issue we are taking our primary application server offline for a brief reboot. Your FoxyCart Store will be unavailable for 5-10 minutes during this window. We apologize for any inconvenience and endeavor to get things back to normal as quickly as possible.

Update January 12, 2012 21:08 PST: Our maintenance window has been completed successfully and all systems are normal at this time.

 

Tuesday, Jan 10th #

2012-01-10 9:00am10:00am -07:00 (PST) 2012-01-10 16:002012-01-10 17:00 +00:00 (GMT)

Mail delay

Impacted Systems: app, admin

Outbound mail was delayed during a 10 minute window this morning due to an issue with our email provider. At this time mails are once again delivering as rapidly as normal.

 

Sunday, Jan 8th #

2012-01-08 10:00pm11:00pm -07:00 (PST) 2012-01-09 05:002012-01-09 06:00 +00:00 (GMT)

Emergency Maintenance: www

Impacted Systems: www, forum, wiki, affiliate

We have finished provisioning a more powerful server stack to replace our venerable www server. This maintenance window is complete the switchover to the new servers. During this time www, forum, and our wiki will be unavailable. Your FoxyCart store will not be affected by this transition.

Update January 7, 2012 11:10 PM: Our maintenance has been completed successfully.

 

Sunday, Jan 8th #

2012-01-08 12:00pm2:00pm -07:00 (PST) 2012-01-08 19:002012-01-08 21:00 +00:00 (GMT)

Planned Maintenance: Forum upgrade

Impacted Systems: forum

We traced our www server’s performance problems to our forum software. We’ve tested out a new version and will be upgrading the forum this weekend. Forums will be unavailable during this maintenance window but FoxyCart stores and documentation will continue to function as normal.

Update January 8, 2012 12:32 PST: Our new forums are rocking and rolling! Your local DNS will update shortly.