[Veyepar] JSON schedule output into website

[parx]

Brief explanation

Zookeepr / Symposium (PyCon Website) JSON schedule output into website + veyepar

Feed validator!!!!

Problem

The Steaming UI and Processing videos relies on the talk schedule. This data is typically in the conference website. There is sometimes an API available,
These APIs are not always accurate, example they may be missing the Keynote entries, or the start time is on a date that is not part of the conference.

Things that will help:

• A spec defining what data is needed.
• A validater to inspect provided data. (command line to validate a local file, public facing web page that will let conference web site developers test their API.)
• Work with the open source conference site projects to add the API to their codebase.

There are 3 popular conference systems:

http://zookeepr.org - https://github.com/zookeepr/zookeepr
http://pentabarf.org - https://github.com/nevs/pentabarf
http://eldarion.com/symposion/ - https://github.com/pinax/symposion

All 3 have schedule exports - examples:
http://www.pytennessee.org/api/schedule_json/
http://lca2013.linux.org.au/programme/schedule/json
https://fosdem.org/2014/schedule/xml

These two systems consume it:
https://github.com/timvideos/streaming-system
https://github.com/CarlFK/veyepar/blob/master/dj/scripts/addeps.py

A description of the data is at the top of addeps.py
and a little more http://nextdayvideo.com/page/metadata.html

https://github.com/CarlFK/veyepar/blob/master/dj/scripts/addeps.py#L288
stores the data to the model
https://github.com/CarlFK/veyepar/blob/master/dj/main/models.py#L228

The reason addeps.py is 2200 lines long is because the conference system coders keep changing things. mostly it is because the symposium system doesn't have the code, so each time someone installs it they write their own. and things evolve too. like adding "what kind of license is it released under?"

symposium team has not accepted the pull request that will help stabilize the API
pinax/symposion#45

Contacts

• Potential Mentors: TimVideos/Veyepar team

[ghost]

Hello,

I am Pavel Nicolae Teodor from University POLITEHNICA of Bucharest. I am looking forward to contribute towards organziation. I am new to open source and need some guidance for same. .I'm good at C/C++ but i can write code in python too . Would you like to give me some guidance on this project

[CarlFK]

http://code.timvideos.us/Summer-Of-Code.html

I added some more details to the item you replied to.

On Sun, Mar 9, 2014 at 3:00 PM, teo721 [email protected] wrote:

Hello,

I am Pavel Nicolae Teodor from University POLITEHNICA of Bucharest. I am
looking forward to contribute towards organziation. I am new to open source
and need some guidance for same. .I'm good at C/C++ but i can write code in
python too . Would you like to give me some guidance on this project

Reply to this email directly or view it on GitHubhttps://github.com//issues/16#issuecomment-37137082
.

Carl K

[micolous]

Unfortunately I'm providing some information a little late for GSoC 2014, but some useful information I think none the less, if someone wants to work on this.

I have several open issues with Zookeepr's (as used by Linux.conf.au above) JSON schedule output which may cause you problems.

You may want to look at fixing these or coordinating with upstream to have these fixed, because it will greatly improve the quality of the data that you receive.

• Start times in schedule JSON are not ISO8601 (defacto JSON date format), and are expressed in localtime with no timezone information zookeepr/zookeepr#270 : Start time expressed as local time without annotation; not using ISO8601 formatting on start time or duration (using datetime.__str__ and timedelta.__str__ instead).
• No miniconf information in schedule JSON or iCalendar feed zookeepr/zookeepr#271 : No annotations about miniconf streams (meaning you won't know which miniconf an event belongs to)
• Add tags to schedule JSON for special events like meals, keynotes, private events for machine parsing zookeepr/zookeepr#272 : No annotations about keynotes, meals, private events (which shouldn't be recorded)

Symposion (as used by PyTennessee above) seems to output all of it's datetime values as ISO8601 but doesn't include timezone information, and it's not clear how it tags special events or miniconf tracks.

Pentabarf (as used by Fosdem above) outputs it's datetime values split up across attributes of multiple tags, does include miniconf information (tracks) and special event tagging. It doesn't include any timezone information in it's output either.

When you (potential assignee) want to look at solving this problem, you'll want to make sure:

• Common event types that shouldn't be recorded can easily be tagged as such in a structured way (eg: {'title': 'Conference Breakfast', 'recorded': false}) This sometimes happens for talks that are given by people who work for a government intelligence agency. This means you don't have to pick through a list of common tags and guess.
• All the datetime values are expressed in ISO8601 format, with timezone metadata. In practice this will probably mean you should have the conference scheduling system convert those values to UTC first, and maybe include an attribute for what Olson timezone it is held in.
• Express times as start + end time, rather than using duration. This can be calculated anyway, and makes it easier to do database selections based on times.

[mithro]

This issue is also relavent timvideos/streaming-system#68

[micolous]

Another speedhump with Zookeepr not outlined above is that different events tend to fork it every 12 months, work on it on their own, and then very occasionally merge back into master. This makes tracking Zookeepr extremely hard, because each conference will change things in subtle ways.

The good news, though, is that there's a new effort (in the last few months) to reconcile these event specific forks back to ZK master, and make the codebase much more accessible to new developers. 👍

Hopefully by the start of GSoC 2016, Zookeepr will be in a better shape, which should sort out most of the Australian and New Zealander events.

One other side-course you may take is to push each of them to expose iCalendar files, and add a light amount of sugar to these iCalendar files for consumption by veyepar. iCalendar is a standard format, our own JSON feeds are not, and it allows any conference software to at least get the basics down in a standard way.