Open Media Network spec

Spec: RSS in for content aggregation#

aka the **Open Media Network**#

1. Overview**#

1. Intended Audience

Prospective coders, mentors, and managers of this project.

Project manager is Richard Hering

richard@visionon.tv 07894350478 skype: richarddirecttv

2. Project Scope and Product Context

RSS in and out by multi-tagging

Scope: Needs to poll a number of RSS items to bring RSS feeds into the liferay database as web content items.

Context: To be used by the visionOntv project (http://visionon.tv), and as an exemplar for other projects to do aggregation.

An aspiration for this tool as an exemplar (NOT part of this spec) is that, in future development, it will create content feeds using boolean logic by tag (rather than the whole site) to input and output user feeds. This would empower the producer of the content rather than the techie at the centre to influence where content ends up, by tagging. With uptake from other open video nodes, we can then build a powerful Open Video Network. NOTE: this is an aspiration, giving more a sense of purpose, NOT part of this current spec!

3. References and Further Reading

http://www.liferay.com/community

2. Detailed Environmentals and Constraints**#

1. Users

Initially, the core visionOntv crew, who are non-techie. Then, people on other projects who need to be able to implement the same functionality in their own cms. The application will also be released back into the liferay community as an open source solution.

2. Operating Environment

Hosted on a web server – could be in a cloud if on Amazon (its open source implementations) – needs to run independently, not tied into corporate structures.

3. Pre-existing Design/Implementation Constraints

It MUST work with liferay standards.

Language:

Preferably java, as liferay is java-based, but open to any suggestions. Specific tools or technologies? Based on existing open, industrial standards such as xml and rss.

4. Assumptions and Dependencies

Code has to be understood by other people – not spaghetti code. As we want to open source the code, it must be readable and developable by other coders.

3. Detailed Requirements**#

















--

Requirement Reference: REQ-1#

Title: RSS aggregator - first version

1. Description and Priority

A basic web application which polls rss feeds to find if they have updated. If the feed has updated, it imports the feed into the database, and then de-dupes it against what is already in the database. It then needs to inject the rss item into the liferay database as formatted web content using the REST API.

High priority: We need to get this basically working on our dev server to test the liferay workflow.

2. Stimulus/Response Sequences

Need a UI consisting of email + password + CAPTCHA for admin login.

Need a UI to add and remove rss feeds, and add the video producer's website. This site will likely be different from the hosting site of the video.

3. Functional Requirements

For REQ-1, if a feed is invalid, we need an error report to say why.

4. Non-Functional Requirements

a) Requirement References

NFR-1: Performance

NFR-2: Security

NFR-3: Quality

b) Description and Priority

NFR-1: Performance

Low priority. No risk because only tested on development server.

NFR-2: Security

Low priority. No risk because only tested on development server.

NFR-3: Quality

Low priority. No problem at this stage, and only tested on development server.



















Requirement Reference: REQ-2**#

Title: Robust error checking for imported rss feeds.

1. Description and Priority

Robust dealing with errors so that the application recovers / keeps running. High priority.

Penalty 9/10 - cannot stop as a result of an error message

Benefit 9/10 - no downtime.

2. Stimulus/Response Sequences

Needs to log errors and email critical errors to admins. Needs a button to display the error log in a web browser.

3. Functional Requirements

For REQ-2, if a feed is invalid, we need an error report to say why.

4. Non-Functional Requirements

a) Requirement References

NFR-1: Performance

NFR-2: Security

NFR-3: Quality

b) Description and Priority

NFR-1: Performance

5/10: Robustness and reliability are more important than real time performance in this version.

NFR-2: Security

5/10: Reasonable, not perfect.

NFR-3: Quality

3/10. Error messages may be cryptic, but ok at this stage.

















--

Requirement Reference: REQ-3**#

Title: Basic editing of field matching

1. Description and Priority

Basic editing of field matching to ensure good formatting of liferay web content.

High priority: We need this to deal with glitches caused by use of different fields by different rss feeds.

2. Stimulus/Response Sequences

Ideally, a drag-and-drop UI which has a two-column view of source items from an rss feed in one column and our liferay web content in the other. Deal with mis-matches by dragging the field from the source rss feed to where it should beon the liferay web content feed, so that corresponding fields match.

3. Functional Requirements

Basic editing of field matching to ensure good formatting of liferay web content. It is possible that our main video feeds will do the matching already, and that an an off-the-shelf piece of software already has this functionality....TBD

4. Non-Functional Requirements

a) Requirement References

NFR-1: Performance

NFR-2: Security

NFR-3: Quality

b) Description and Priority

NFR-1: Performance

Low priority. Only done once per feed, therefore not much hit on the system.

NFR-2: Security

Medium priority. Eg trojans in the rss feeds. App needs to be able strip out suspicious fields therefore.

NFR-3: Quality

High priority. Don't want dodgy stuff in our database, and would be a real hassle to repair afterwards.
















Requirement Reference: REQ-4**#

Title: Make de-duping cleverer

1. Description and Priority

Make the de-duping cleverer, so as to apply more checks of copies - to deal with spam and re-aggregation.

High priority: Or the database can become a dustbin!

2. Stimulus/Response Sequences

No UI needed TBD

3. Functional Requirements

Make the de-duping cleverer, by GUID, url and title. If the item fails 2 out of 3, it gets held in a moderation queue, and admins get an error message. If it fails all three, we don't even get an error message. TBD

4. Non-Functional Requirements

a) Requirement References

NFR-1: Performance

NFR-2: Security

NFR-3: Quality

2. Description and Priority

NFR-1: Performance

High priority. Needs to be optimised for strain on server.

NFR-2: Security

High priority. Need to be creative about implementing security so that it can't get into a loop and seize up. (But the de-duping should deal with this already!)

Possible solution: restrict the number of items within a certain timeframe. TBD

Risk 9/10: If de-duping fails, could get into a loop, fill up liferay with masses of spam content, and crash the server. Problem of another aggregator.

NFR-3: Quality

High priority. Needs to 98% successful.

















Requirement Reference: REQ-5**#

Title: Security

1. Description and Priority

Security: app needs to be locked down so it only has the ability to perform its role of injecting web content. It must not interfere with the production server's ability to serve pages to users, must be secured against non-authorised users. TBD

2. Stimulus/Response Sequences

App needs to written in such a way that it is resistant to doing anything else but inject the web content.

3. Functional Requirements

This implementation must interfere with neither the basic functionality, nor with the user experience and reliability.

4. Non-Functional Requirements

a) Requirement Reference

NFR-1: Performance

NFR-2: Security

NFR-3: Quality

b) Description and Priority

NFR-1: Performance

High priority. Needs to be robust.

NFR-2: Security

(Tautology)

NFR-3: Quality

High priority. Needs to be effectively locked off.

















FURTHER VERSIONS - Optimisation and making real time**#

Look at real-time scaling with Pub Sub Hubbub and fat pings to see if they are a good idea and give us feedback.

Check that liferay can do real time feed caching.

(Optional) Need a UI to set the polling time for different feeds.

0 Attachments
6902 Views
Average (0 Votes)
The average rating is 0.0 stars out of 5.
Comments

Flattr this

October 17
18:43
Richard Hering updated a blog entry, Resistance Archive app.
18:04
Richard Hering wrote a new blog entry, Resistance Archive app.
17:19
August 29
FSDFS commented on a wiki page, OpenPlayer mobile app.
11:13
April 27
Hamish Campbell updated a wiki page, Newsflash.
17:23
Hamish Campbell updated a wiki page, Newsflash.
15:28
Hamish Campbell updated a wiki page, Newsflash.
15:27
February 15
Harry Wood wrote a new message board post, OpenStreetMap London.
July 24
Richard Hering updated a wiki page, OpenPlayer mobile app.
17:06
July 22
Richard Hering updated a wiki page, OpenPlayer mobile app.
13:38
Subscribe to springofcode's activities. (Opens New Window)