2 articles Tag collaboration

IPython: interactive/self-documenting data analysis

by on September 9, 2013, 1 comment

IPython is an “interactive” framework for writing python code. Code snippets can be run at the programmer’s will and the output will be displayed right below the code. Together with rich input from html-markup to iFrames, an entire workflow can be fully documented. This is very handy for learning, of course, but also to make a complex analysis of a computer incident available and transparent to later readers. As everything (docu, code, output) gets “statically” saved in JSON, the documentation is even independent of the availability of data sources. (Note: there is also a special “Notebook viewer” available online so the reader doesn’t have to know/have IPython her/himself)

As a couple of powerful viz and analysis libraries are available for Python (such as PANDAS), this is (almost) ideal for recording an analysts way to a result.

Ideas for improvement:

  1. make it even more interactive/auto-updating so that changes in one place (“cell”) show up in other places at once (maybe even work with realtime sources?) – maybe towards frameworks like puredata/MAX: this would help explore various parameters for the analysis functions.
  2. Think about some auto-recording functions so that documentation becomes easier and the “author” has to think less about it. This might be especially possible in the narrow context of network security analysis where certain procedures are standardized or very common.

See how it works, e.g. with PCAPS (German)

Thanks to Genua who shared their internal training so well recorded and so generously!

Tags: , , , ,

Inside AT&T Network Operation Center

by on June 2, 2013, 0 comments

Every time we go online, make a phone call, send an SMS, we use the networks of large operators. These are large technical constructions and they need permanent monitoring and maintenance to work as we expect (which is: we don’t notice they are even there).

Network Operations Centers (NOC) are the institutions where network operators concentrate experts and technology to permanently check parameters of the networks, fix problems, and detect malfunctions and malware. Through their unique position, these NOCs are usually heavily shielded from the outside world.

This video gives a short insight into the Global NOC of AT&T (Bedminster, NJ), including a glimpse on their visualisations and an interview with Chuck Kerschner (Director of Network Operations at AT&T).

Friedmann and Kerschner in front of the video wall of the AT&T GNOC (click image for video)Friedmann and Kerschner in front of the video wall of the AT&T GNOC

Although Lex Friedman of TechHive asks the “right questions” (i.e. the questions we have as well), the answers are often a bit short and too general to learn a lot from them. Still, an interesting video for inspiration.

View on the large shared dashboard at AT&T (in the video at 1:20)View on the large shared dashboard at AT&T (in the video at 1:20)

A little more detais are availble here as audio, and in an WSJ article about a specialist working at AT&T to prepare for unusual traffic spikes.

Even closer to the SASER/Siegfried project are (Information) Security Operations Centers (SOCs) – note that Kerschner is mostly concerned with storms or technical outages, not with security threats like viruses or botnets. Steve Roderick is the colleague at the AT&T center responsible for security.

 

Tags: , , , , ,