Manipulating VOEvents in code¶
We’ve seen that VOEvents are just text documents, and you can happily read and write them in your favourite text editor. That’s fine as far as it goes, but it’s not a practical approach for building a large scale event handling system. Luckily, one of the advantages of using XML to encode VOEvents is that we can tap into a wide range of infrastructure and tools which have been developed specifically for processing XML. Effectively every mainstream programming language has a whole suite of libraries for handling XML available—see, for example, the documentation for the Python standard library.
While it’s possible to directly apply these generic XML handling tools to VOEvents, it’s even more convenient to use a library which has been especially developed to work with VOEvents. Here, we are using voevent-parse which does just that. Voevent-parse builds on the popular lxml library.
The examples below should be just enough to get you started with voevent-parse. An expanded version which goes into a little more detail on the syntax quirks of the lxml library is also available in IPython Notebook format as the voevent-parse-tutorial, In addition, we refer you to lxml’s documentation for full details.
Parsing a VOEvent¶
Let’s start by reading some basic data from a VOEvent. We’ll save the example
we looked at earlier to a file—say,
voevent.xml—and read that into Python. You can follow along using
First, we import the
voeventparse library into Python, and use it to load
voevent.xml file from disk:
In : import voeventparse as vp In : with open('voevent.xml') as f: ...: v = vp.load(f) ...: In : v Out: <Element VOEvent at 0x104747560>
The basic “attributes” (the
role and the
ivorn, as well as some of the
XML boilerplate that we skipped over) of the root
VOEvent element are accessible as a
In : v.attrib['ivorn'] Out: 'ivo://org.hotwired/exciting_events#123' In : v.attrib['role'] Out: 'test'
We can also descend into the sub-elements of the event and retrieve the key information:
In : v.Who.Author.title Out: 'Hotwired VOEvent Tutorial' In : v.What.Group.Param.Description Out: 'Peak Flux' In : v.What.Group.Param.attrib['value'] Out: '0.0015'
Wait a minute—there are actually two
Param elements inside
do we get the integrated flux? Two possibilities, actually. First, you can
iterate over the children of the parent element:
In : for element in v.What.Group.iterchildren(): ...: print element.attrib['name'], ": ", element.attrib['value'] ...: peak_flux : 0.0015 int_flux : 2.0e-3
Alternatively, you can search for particular elements:
In : v.find(".//Param[@name='int_flux']").attrib['value'] Out: '2.0e-3'
It’s worth noting that the VOEvent schema does not guarantee that a
VOEvent will provide a
Param with a name of
int_flux. The flexibility
of VOEvent is both a blessing and a curse here: since different sources of
events will likely report on quite different types of observation, it’s hard
to make sense of them without already having some idea of what you expect to
find inside. The good news is that you’ll probably have quite a good idea of
which event streams you’re interested in, and you can expect (although it’s
not guaranteed...) events with in a particular stream to follow a regular
On the subject of the schema: you’ll come across all sorts of weird constructs that people have stashed in VOEvents, and you might even be tempted to come up with some yourself. Often, folks will be able to figure out what you mean, and XML handling tools generally try to be quite liberal in what they’ll accept. However, the schema exists for a reason: if you stray outside its boundaries, all bets are off regarding whether your events will be interpreted in the way you intend. Indeed, many systems will simply refuse to accept an event which isn’t valid according to the schema. Conveniently, voevent-parse can check this for you:
In : vp.valid_as_v2_0(v) Out: True
Note that we are checking against the VOEvent version 2.0 schema, which is the latest version and is definitely what you should be using.
Creating a VOEvent¶
Having seen how we can use voevent-parse to read a VOEvent, the next step is to create one. In order to make this a bit more interesting, let’s use a real transient. Gaia has recently started publishing Photometric Science Alerts to a publicly-accessible website, but they aren’t yet available as VOEvents. Let’s pick on the following:
|Name||Observed||Published||RA (deg.)||Dec. (deg.)||Magnitude||Historic mag.||Historic scatter||Class||Comment|
|Gaia14adi||2014-11-07 01:05:09||2014-12-02 13:55:54||168.47841||-23.01221||18.77||19.62||0.07||unknown||Fading source on top of 2MASS Galaxy (offset from bulge)|
We’ll start by creating the skeleton of our VOEvent packet. We carefully to
set the role to
test so that nobody is tempted to start acting on the
contents of this demo event. We also set the timestamp in the
Who block to
the time the event was generated (not when the observation was made), as per
In : import voeventparse as vp In : import datetime In : v = vp.Voevent(stream='hotwired.org/gaia_demo', stream_id=1, role=vp.definitions.roles.test) In : vp.set_who(v, datetime.datetime.utcnow())
Now to define the author. Note that this is us, since we’re generating the VOEvent—this isn’t an official Gaia product, and we neither want to claim credit for the result ourselves, nor do we want people to start hassling the Gaia folks with questions about our event. We’ll make sure that’s noted in the explanatory text attached to the event:
In : vp.set_author(v, title="Hotwired VOEvent Hands-on", contactName="John Swinbank") In : v.Description = "This is not an official Gaia data product."
Now let’s add details of the observation itself. We’ll record both the magnitude that Gaia is reporting for this particular event, and the historic values they also provide:
In : v.What.append(vp.Param(name="mag", value=18.77, ucd="phot.mag")) In : h_m = vp.Param(name="hist_mag", value=19.62, ucd="phot.mag") In : h_s = vp.Param(name="hist_scatter", value=0.07, ucd="phot.mag") In : v.What.append(vp.Group(params=[h_m, h_s], name="historic"))
Now we need to specify where and when the observation was made. Rather than
trying to specify a position for Gaia, we’ll just call it out by name. Note
that Gaia don’t provide errors on the position they cite, so we’re rather
In : vp.add_where_when(v, coords=vp.Position2D(ra=168.47841, dec=-23.01221, err=0, units='deg', system=vp.definitions.sky_coord_system.fk5), obs_time=datetime.datetime(2014, 11, 7, 1, 5, 9), observatory_location="Gaia")
We should also describe how this transient was detected, and refer to the name that Gaia have assigned it. Note that we can provide multiple descriptions (and/or references) here:
In : vp.add_how(v, descriptions=['Scraped from the Gaia website', 'This is Gaia14adi'], references=vp.Reference("http://gsaweb.ast.cam.ac.uk/alerts/"))
Finally, we can provide some information about why this even might be scientifically interesting. Gaia haven’t provided a classification, but we can at least incorporate the textual description:
In : vp.add_why(v) In : v.Why.Description = "Fading source on top of 2MASS Galaxy (offset from bulge)"
Finally—and importantly, as we discussed above—let’s make sure that this event is really valid according to our schema:
In : vp.valid_as_v2_0(v) True
Great! We can now save it to disk:
In : with open('gaia.xml', 'w') as f: vp.dump(v, f)
And we’re all done. You can open the file in your favourite editor to see what
we’ve produced, but note that it probably won’t be particularly elegantly
formatted. You can use a tool like
xmllint to pretty print it; you should
end up with something like:
<?xml version="1.0" encoding="UTF-8"?> <voe:VOEvent xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:voe="http://www.ivoa.net/xml/VOEvent/v2.0" xsi:schemaLocation="http://www.ivoa.net/xml/VOEvent/v2.0 http://www.ivoa.net/xml/VOEvent/VOEvent-v2.0.xsd" version="2.0" role="test" ivorn="ivo://hotwired.org/gaia_demo#1"> <Who> <Description>VOEvent created with voevent-parse: https://github.com/timstaley/voevent-parse</Description> <Date>2015-04-17T15:52:56</Date> <Author> <title>Hotwired VOEvent Hands-on</title> <contactName>John Swinbank</contactName> </Author> </Who> <What> <Param dataType="float" name="mag" ucd="phot.mag" value="18.77"/> <Group name="historic"> <Param dataType="float" name="hist_mag" ucd="phot.mag" value="19.62"/> <Param dataType="float" name="hist_scatter" ucd="phot.mag" value="0.07"/> </Group> </What> <WhereWhen> <ObsDataLocation> <ObservatoryLocation id="Gaia"/> <ObservationLocation> <AstroCoordSystem id="UTC-FK5-GEO"/> <AstroCoords coord_system_id="UTC-FK5-GEO"> <Time unit="s"> <TimeInstant> <ISOTime>2014-11-07T01:05:09</ISOTime> </TimeInstant> </Time> <Position2D unit="deg"> <Name1>RA</Name1> <Name2>Dec</Name2> <Value2> <C1>168.47841</C1> <C2>-23.01221</C2> </Value2> <Error2Radius>0</Error2Radius> </Position2D> </AstroCoords> </ObservationLocation> </ObsDataLocation> </WhereWhen> <Description>This is not an offical Gaia data product.</Description> <How> <Description>Scraped from the Gaia website</Description> <Description>This is Gaia14adi</Description> <Reference uri="http://gsaweb.ast.cam.ac.uk/alerts/"/> </How> <Why> <Description>Fading source on top of 2MASS Galaxy (offset from bulge)</Description> </Why> </voe:VOEvent>