[Micronet] Air Bears down during peak hours

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[Micronet] Air Bears down during peak hours

berkeley9241

System Status says --


Scheduled Outage Various Network Services

Posted by CSS IT ~nk 
On 4/22/2013 at 11:44 am PST
Posted in Scheduled Outage

Outage Type: SCHEDULED OUTAGE
Date Submitted: Wednesday, April 24, 2013
Outage Start/End Time: 1900 – 1930
Groups Impacted: Warren Hall (Data Center)
Equipment: alps.ist

Description: The IST-Database group will reboot production PostgreSQL database server alps.ist to update the operating system software and to perform a file system check.

During this outage window the following services will not be available:

AirBears authentication (including guest accounts)
VPN authentication
DHCP registration
AirBears Guest Account Service Request
ARP cache querries
firewall contexts administrator access

Please direct any questions you may have regarding this announcement to: Kevin Wong

CMR: 2081


7:00 PM seems like it would be a peak use time for Air Bears and VPN.  But I don't see any data to show that use by time of day at http://fragment1.berkeley.edu/~cricket/ or http://www.net.berkeley.edu/usage_stats/ to point to.

Can't IS&T do work like this at some time when there are not classes in session and lots of people in the library?  Or does this work really only impact AirBears and VPN in Warren Hall (Data Center)?


[directory.berkeley.edu shows 7 Kevin Wong's on campus so I didn't know which one to send that question to]



 
-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.
Reply | Threaded
Open this post in threaded view
|

Re: [Micronet] Air Bears down during peak hours

Kellie HOBBS
This outage will be rescheduled to a more convenient time after the end of the semester.

We apologize for any confusion that scheduling this outage may have caused.

Kellie Hobbs
IST Database Group


On Tue, Apr 23, 2013 at 9:13 AM, berkeley9241 <[hidden email]> wrote:

System Status says --


Scheduled Outage Various Network Services

Posted by CSS IT ~nk 
On 4/22/2013 at 11:44 am PST
Posted in Scheduled Outage

Outage Type: SCHEDULED OUTAGE
Date Submitted: Wednesday, April 24, 2013
Outage Start/End Time: 1900 – 1930
Groups Impacted: Warren Hall (Data Center)
Equipment: alps.ist

Description: The IST-Database group will reboot production PostgreSQL database server alps.ist to update the operating system software and to perform a file system check.

During this outage window the following services will not be available:

AirBears authentication (including guest accounts)
VPN authentication
DHCP registration
AirBears Guest Account Service Request
ARP cache querries
firewall contexts administrator access

Please direct any questions you may have regarding this announcement to: Kevin Wong

CMR: 2081


7:00 PM seems like it would be a peak use time for Air Bears and VPN.  But I don't see any data to show that use by time of day at http://fragment1.berkeley.edu/~cricket/ or http://www.net.berkeley.edu/usage_stats/ to point to.

Can't IS&T do work like this at some time when there are not classes in session and lots of people in the library?  Or does this work really only impact AirBears and VPN in Warren Hall (Data Center)?


[directory.berkeley.edu shows 7 Kevin Wong's on campus so I didn't know which one to send that question to]




-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.



 
-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.
Reply | Threaded
Open this post in threaded view
|

Re: [Micronet] Air Bears down during peak hours

Patty Mead
Is this just for tomorrow or does this also mean Fridays outage is rescheduled?


On Tue, Apr 23, 2013 at 1:38 PM, Kellie HOBBS <[hidden email]> wrote:
This outage will be rescheduled to a more convenient time after the end of the semester.

We apologize for any confusion that scheduling this outage may have caused.

Kellie Hobbs
IST Database Group


On Tue, Apr 23, 2013 at 9:13 AM, berkeley9241 <[hidden email]> wrote:

System Status says --


Scheduled Outage Various Network Services

Posted by CSS IT ~nk 
On 4/22/2013 at 11:44 am PST
Posted in Scheduled Outage

Outage Type: SCHEDULED OUTAGE
Date Submitted: Wednesday, April 24, 2013
Outage Start/End Time: 1900 – 1930
Groups Impacted: Warren Hall (Data Center)
Equipment: alps.ist

Description: The IST-Database group will reboot production PostgreSQL database server alps.ist to update the operating system software and to perform a file system check.

During this outage window the following services will not be available:

AirBears authentication (including guest accounts)
VPN authentication
DHCP registration
AirBears Guest Account Service Request
ARP cache querries
firewall contexts administrator access

Please direct any questions you may have regarding this announcement to: Kevin Wong

CMR: 2081


7:00 PM seems like it would be a peak use time for Air Bears and VPN.  But I don't see any data to show that use by time of day at http://fragment1.berkeley.edu/~cricket/ or http://www.net.berkeley.edu/usage_stats/ to point to.

Can't IS&T do work like this at some time when there are not classes in session and lots of people in the library?  Or does this work really only impact AirBears and VPN in Warren Hall (Data Center)?


[directory.berkeley.edu shows 7 Kevin Wong's on campus so I didn't know which one to send that question to]




-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.




-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.




--
 
Patty Mead
Assistant Dean - IIT
College of Environmental Design
471 Wurster Hall, MC 1810
Berkeley, California 94720

voice: 510-501-7973
http://ced.berkeley.edu 

 
-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.
Reply | Threaded
Open this post in threaded view
|

Re: [Micronet] Air Bears down during peak hours

Gary Thomas
Patty and Micronetters,

The outage for the very early morning on Friday is a separate event from
the alps database server reboot. The 3am Friday event is an image
upgrade for the infrastructure. It is still on the schedule to be completed.

--Gary Thomas
IST - Wireless Networks

On 4/23/2013 1:48 PM, Patty MEAD wrote:

> Is this just for tomorrow or does this also mean Fridays outage is
> rescheduled?
>
>
> On Tue, Apr 23, 2013 at 1:38 PM, Kellie HOBBS <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     This outage will be rescheduled to a more convenient time after
>     the end of the semester.
>
>     We apologize for any confusion that scheduling this outage may
>     have caused.
>
>     Kellie Hobbs
>     IST Database Group
>
>
>     On Tue, Apr 23, 2013 at 9:13 AM, berkeley9241
>     <[hidden email] <mailto:[hidden email]>> wrote:
>
>
>               System Status says --
>
>
>
>               Scheduled Outage Various Network Services
>
>         Posted by CSS IT ~nk
>         On 4/22/2013 at 11:44 am PST
>         Posted in Scheduled Outage
>         <http://ucbsystems.org/category/active/scheduled-outage/>
>
>         *Outage Type*: SCHEDULED OUTAGE
>         *Date Submitted*: Wednesday, April 24, 2013
>         *Outage Start/End Time*: 1900 – 1930
>         *Groups Impacted*: Warren Hall (Data Center)
>         *Equipment*: alps.ist
>
>         *Description*: The IST-Database group will reboot production
>         PostgreSQL database server alps.ist to update the operating
>         system software and to perform a file system check.
>
>         During this outage window the following services will not be
>         available:
>
>         AirBears authentication (including guest accounts)
>         VPN authentication
>         DHCP registration
>         AirBears Guest Account Service Request
>         ARP cache querries
>         firewall contexts administrator access
>
>         Please direct any questions you may have regarding this
>         announcement to: Kevin Wong
>
>         *CMR*: 2081
>
>
>         7:00 PM seems like it would be a peak use time for Air Bears
>         and VPN. But I don't see any data to show that use by time of
>         day at http://fragment1.berkeley.edu/~cricket/
>         <http://fragment1.berkeley.edu/%7Ecricket/> or
>         http://www.net.berkeley.edu/usage_stats/ to point to.
>
>         Can't IS&T do work like this at some time when there are not
>         classes in session and lots of people in the library? Or does
>         this work really only impact AirBears and VPN in Warren Hall
>         (Data Center)?
>
>
>         [directory.berkeley.edu <http://directory.berkeley.edu> shows
>         7 Kevin Wong's on campus so I didn't know which one to send
>         that question to]
>
>
>
>
>         -------------------------------------------------------------------------
>         The following was automatically added to this message by the
>         list server:
>
>         To learn more about Micronet, including how to subscribe to or
>         unsubscribe from its mailing list and how to find out about
>         upcoming meetings, please visit the Micronet Web site:
>
>         http://micronet.berkeley.edu
>
>         Messages you send to this mailing list are public and
>         world-viewable, and the list's archives can be browsed and
>         searched on the Internet. This means these messages can be
>         viewed by (among others) your bosses, prospective employers,
>         and people who have known you in the past.
>
>
>
>
>     -------------------------------------------------------------------------
>     The following was automatically added to this message by the list
>     server:
>
>     To learn more about Micronet, including how to subscribe to or
>     unsubscribe from its mailing list and how to find out about
>     upcoming meetings, please visit the Micronet Web site:
>
>     http://micronet.berkeley.edu
>
>     Messages you send to this mailing list are public and
>     world-viewable, and the list's archives can be browsed and
>     searched on the Internet. This means these messages can be viewed
>     by (among others) your bosses, prospective employers, and people
>     who have known you in the past.
>
>
>
>
> --
>  
> Patty Mead
> Assistant Dean - IIT
> College of Environmental Design
> 471 Wurster Hall, MC 1810
> Berkeley, California 94720
>
> voice: 510-501-7973
> http://ced.berkeley.edu 
>
>
>  
> -------------------------------------------------------------------------
> The following was automatically added to this message by the list server:
>
> To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:
>
> http://micronet.berkeley.edu
>
> Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.


 
-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.
Gary Thomas
Wireless Network Engineer
Network Services and Operations
Information Services and Technology
University of California Berkeley
Email: gjthomas@berkeley.edu
Voice: 510-643-0370
Fax: 510-642-5622
Reply | Threaded
Open this post in threaded view
|

Re: [Micronet] Air Bears down during peak hours

Rune Stromsness-2
In reply to this post by berkeley9241
On 23-Apr-13 09:13, berkeley9241 wrote:

>       System Status says --
>
>       Scheduled Outage Various Network Services
>
> Posted by CSS IT ~nk
> On 4/22/2013 at 11:44 am PST
> Posted in Scheduled Outage
> <http://ucbsystems.org/category/active/scheduled-outage/>
>
> *Outage Type*: SCHEDULED OUTAGE
> *Date Submitted*: Wednesday, April 24, 2013
> *Outage Start/End Time*: 1900 – 1930
> *Groups Impacted*: Warren Hall (Data Center)
> *Equipment*: alps.ist
>
> *Description*: The IST-Database group will reboot production PostgreSQL
> database server alps.ist to update the operating system software and to
> perform a file system check.
>
> During this outage window the following services will not be available:
>
> AirBears authentication (including guest accounts)
> VPN authentication
> DHCP registration
> AirBears Guest Account Service Request
> ARP cache querries
> firewall contexts administrator access
>
> Please direct any questions you may have regarding this announcement to:
> Kevin Wong
>
> *CMR*: 2081
I believe that Kellie from the IST Database team has already responded
with some specific information about re-scheduling this week's maintenance.

A bit more background:  the network team is just one of many customers
who use the open source database service from the IST Database team.  We
provide them with input about good or bad times from our perspective,
but we rely on them to decide on the maintenance windows for their
service taking the impact to all of their customers into account.

> 7:00 PM seems like it would be a peak use time for Air Bears and VPN.
>  But I don't see any data to show that use by time of day
> at http://fragment1.berkeley.edu/~cricket/ or http://www.net.berkeley.edu/usage_stats/ to
> point to.

We don't have any publicly available direct graphs showing use of
AirBears or the VPN, but with a little bit of sleuthing in traceroute
output or knowledge of the design you can get a few graphs that show
basic traffic levels for both services.  I'll include links to those at
the end of this email.  [1]

Generally the VPN sees use with no clear "quiet" time throughout a
normal 24-hour period (which tends to make some sense for something that
UC Berkeley faculty, staff, and students can use from anywhere in the
world).

While classes are in session AirBears peak use is generally the
afternoon hours (roughly 1 pm to 6 pm), while lowest use is generally 2
am to 7 am (which also tends to make sense for a service that one can
only use while physically located in or near a UC Berkeley building).

> Can't IS&T do work like this at some time when there are not classes in
> session and lots of people in the library?  Or does this work really

When the network group is scheduling maintenance we tend to work for a
balance between the time that would cause the least impact to campus and
the ability network and other staff to perform the maintenance and
handle any unexpected fallout.  For key pieces of the campus network
that often means scheduling work for weekdays in the early morning.
(For instance, you can see also on systemstatus.berkeley.edu right now
another AirBears maintenance window that Patty and Gary have just
discussed scheduled by our team for 3 am - 7 am on Friday morning,
during the lowest use time that also gives us normal support resources
in the hours after the work to work through any unintended problems.)

> only impact AirBears and VPN in Warren Hall (Data Center)?

The scheduled work that is referenced at the start of this email would
have impacted all AirBears and VPN use, not just the use in Warren Hall.
 Our standard template for announcing network outages asks the
announcing engineer to provide the building.  Our group will be
discussing soon how to use that field to most clearly let people reading
our notices know what the impact will be.

> [directory.berkeley.edu <http://directory.berkeley.edu> shows 7 Kevin
> Wong's on campus so I didn't know which one to send that question to]

The Kevin Wong in our group has tried to make it easier for people to
find him in the directory by making his last name in the campus
directory "Wong, IST-IS".

Some almost-current information about how we schedule outages is
available at:
http://ist.berkeley.edu/node/484/

That page also talks about the [hidden email]
mailing list where we announce network maintenance and problems.  On
that list we provide more specific email addresses for the engineer
responsible for network work, but that information is removed from
systemstatus.berkeley.edu to reduce the amount of spam that people send
to addresses posted on public websites.


Rune



[1]
As of today, production AirBears traffic currently uses 5 different
links from AirBears routers to the campus network core.  Graphs of
traffic on each of those individual links are available at:
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-201%2Fgigabitethernet3_9;ranges=d%3Aw%3Am%3Ay;view=Octets
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-202%2Fgigabitethernet3_44;ranges=d%3Aw%3Am%3Ay;view=Octets
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-202%2Ftengigabitethernet6_5;ranges=d%3Aw%3Am%3Ay;view=Octets
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-202%2Fgigabitethernet4_45;ranges=d%3Aw%3Am%3Ay;view=Octets
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-201%2Fgigabitethernet3_45;ranges=d%3Aw%3Am%3Ay;view=Octets

As of today traffic for the campus Remote Access VPN uses the link with
traffic graphed at:
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-240%2Fgigabitethernet3_36;ranges=d%3Aw%3Am%3Ay;view=Octets


>
>  
> -------------------------------------------------------------------------
> The following was automatically added to this message by the list server:
>
> To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:
>
> http://micronet.berkeley.edu
>
> Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.
>
--
Rune Stromsness
Network Operations & Services
Telecommunications
University of California, Berkeley
[hidden email]


 
-------------------------------------------------------------------------
The following was automatically added to this message by the list server:

To learn more about Micronet, including how to subscribe to or unsubscribe from its mailing list and how to find out about upcoming meetings, please visit the Micronet Web site:

http://micronet.berkeley.edu

Messages you send to this mailing list are public and world-viewable, and the list's archives can be browsed and searched on the Internet.  This means these messages can be viewed by (among others) your bosses, prospective employers, and people who have known you in the past.

signature.asc (269 bytes) Download Attachment