Skip to content
This repository has been archived by the owner on Aug 26, 2019. It is now read-only.

Previous Wiki

Jump to bottom
Zoé McLennan edited this page May 31, 2017 · 3 revisions

Installations and Instructions

Neume Editor ONline.

Neon.js is a browser-based music notation editor written in JavaScript. The editor can be used to manipulate digitally encoded early musical scores in square-note notation.

Requirements

Running the server

cp conf.py{.dist,}
# edit conf.py and set MEI_DIRECTORY somewhere writable
python server.py [port]

Visit the web application at http://localhost:8080

Development

Documentation is provided by jsdoc-toolkit: http://code.google.com/p/jsdoc-toolkit/

Build docs with (java required)

make doc

Tests

We're using qunit for tests. Load test/neontest.html in a browser to run them

To run server tests, install python-nose and run nosetests

Meetings

Meeting (November 28, 2011)

High Level Requirements

  1. Create training (search/musical edition) data
    • Neume shape
    • Pitch
    • Physical position on the page (musical content)
  2. Manipulations
    • (CRUD) create, read, update, delete (insert, delete, substitute) -> musical content only
  3. Preserve metadata (whatever it reads in and doesn't understand should be passed through)
  4. Display input (don't have edit functionality) start here
    • ie., take liber data and view it in Neon.js
  5. As a neume editor,
    • 4 line staff
    • Solesmes style notation (square-note notation)
    • Quilismas
    • Liquescence
    • Clefs
    • Episemata: horizontal and vertical lines not important for now

Future Requirements

  1. Lyrics
  2. Raw entry (as score editor - intelligent spacing engine)
  3. Episemata (gravy)
  4. <x,y> on neume component

Other Notes

  1. Design code intelligently to allow it to be extensible to common music notation later on; however, must be usable for editing neume notation first and foremost.
  2. Currently in MEI,
(neume component) ... 3. Straight staff lines (then transform it to page coordinate space - including transformations - when rendering) * allow staff lines to be spaced unevenly (bounding box divided by number of lines) when programming make general to allow this * allow non-uniform systems (wherever MEI says they are)

First Steps

  1. Modify python script to add element attributes to parent
  2. Server to serve JSON documents
  3. Make sure javascript can read the compressed JSON (check that JSON file is decompressed when using jquery .get() through ajax and can still access data)
  4. Create skeleton editor page on server
  • Create canvas inside jquery plugin so fabric.js calls are private, import/export JSON functions are public to the extension
  1. Set up google closure to modularize javascript code
  2. Display first system (create algorithm to render staff lines to meet the neumes inside it)
  3. Convert festadiesa_export.ttf to .svg then import through fabric.js (as individual SVG images)

Meeting (December 16, 2011)

Next steps over the holidays

  1. Page and staff model
  2. Draw staves
  3. Read MEI, construct models, draw staves (scaling factor)
  4. Model for neumes
  5. Draw neumes on staff
  6. Read MEI, construct models, draw neumes on staves

JSON vs. XML

  • Select neume & change pitch. Get by ID
    • jQuery: $("#_");
    • jSON: walk data structure -> cache?
  • Delete neume
  • Add part to compound neume (explode)
  • Insert neume (compress, headshape)
  • Insert additionals (bar, episema, dots)

Preliminary Work

This page will overview pros and cons of possible technologies for the client side medieval notation editor, NEON.js.

A primary goal of the Single Interface for Music Score Searching and Analysis (SIMSSA) project is to provide a publically available web application that can perform musical notation editing, recognition, and recognition correction of older musical notation systems. NEON.js is a step in this direction.

Aruspix to SVG

Aruspix is a C++ application that, among many other features, performs optical music recognition and offers a viewer and music editor for visualisation of the results. On the back end, Aruspix uses libMEI to create and maintain a representation of the transcription that conforms to the Music Encoding Initiative (MEI) 2012 schema.

Given that Aruspix is a desktop application, a client-server architecture is necessary in order to make the editor web accessible. In other words, the javascript editor would have to send score modification information to Aruspix on the server, which would process the request (update the libMEI data structure) and create a response for the client to process.

Pros Cons
Documented and unit tested code base to build off. Laurent recently modified the system design to make the code less obfuscated. Still a complicated code base. Lengthy development time.
Scalable Vector Graphic (SVG) rendering exists within Aruspix for common music notation. Would have to be modified for medieval notation. SVG rendering implementation incomplete even for common music notation. Laurent mentioned in an email that, "I did not update the SVG output format, ... there is no system or layer, for example."
Beginning code base for neume editor versus a brand new project. Neume editor within Aruspix is still in beginning stages of development.
In the end, Aruspix will generate a valid MEI document from the maintaned libMEI data structure in memory. MEI import/export for medieval notation has not been implemented yet in the Aruspix editor.
Each note movement or score modification on the client must be sent back to Aruspix running on the server to do sanity checks, update the data model, and possibly rerender the SVG file to send back to the client. Potential for large latency of a server response.
Double the work? Must implement the neume editor within Aruspix, followed by the javascript editor on the web. The problem is that the javascript editor will need sufficient musical knowledge (ie., moving note on the y-axis must snap to valid pitch locations) to operate efficiently. We could implement a musically unaware frontend, responsible only for graphical manipulations, and then send coordinates to the server to do the sanity checks; implemented in this way, the lag will likely become unmanageable.
Client-server architecture increases system complexity. Harder to debug.
Have to host Aruspix on a server and write a CGI wrapper to receive incoming requests and route them to Aruspix.

Javascript SVG Libraries Investigated

  • jQuery SVG
    • (+) Allows groups and group transformations.
    • (+) Allows SVG to be loaded inline or from a file.
    • (-) No click and drag functionality built into the library. Would have to be written from coordinates received from onClick events.
  • Raphaël
    • (+) Easy to use. Fast development.
    • (+) Click and drag functionality. Many animations built in.
    • (-) SVG import not natively available in the library. Open source library available to import SVG (raphaël-svg-import), but is not complete for our purposes (in README, "TODO: SVG group to Raphaël set conversion").
    • (-) Dragging sets (ie., set containing note head and stem) not natively available in the library.
    • (-) Bug? Can not handle nested transformations (translations, rotations, and scalings).

Known Development Time-sinks

  • Developing protocol for sending information between the client and server.
  • Aruspix editor MEI import/export.
  • Festa dias font clean-up.

Homebrew Canvas/SVG

Pros Cons
Starting fresh allows us to design the system from scratch, instead of being constrained by the architecture of Aruspix. Time must be allocated for initial system design.
Development in javascript is arguably faster than development in C++. There are many libraries written in javascript for user interface commands and events (jQuery, etc.) Since libMEI is not ported to javascript, import/export MEI to render in the editor would have to be implemented.
Since all manipulations are done on the client side, fast and almost instantaneous results.
Directly available online as a web application.
If using SVG, we have a DOM structure which we can easily query using jQuery SVG to form the resulting MEI.

Javascript Canvas/SVG Libraries and Tools Investigated

  • VexFlow
    • (+) Directly converts internal data structure to HTML5 canvas.
    • (+) Keeps a representation of the musical structures in memory on the client side. Access to both graphical and musical information.
    • (+) Keeps a map of note coordinates to easily figure out what is being clicked on.
    • (+) Source code is modularized well. Would only have to touch a few classes (namely, formatter.js, timesignature.js, stavenote.js, note.js) to remove the time signature information. Or we could leave it as is and take an approach similar to Finale medieval editor and dynamically change the time signature information on the fly as notes are added and changed.
    • (-) Currently only supports common music notation. Would have to fork the github repository and modify for medieval notation. Would have to modify the formatter (rendering engine) to not consider time signature information, 4 staff lines, etc.
  • Cloud Composer 5: a very simple (more proof of concept) editor using VexFlow.
    • (+) A good starting point.
    • (-) By no means complete. A lot is hardcoded since this was a hack implemented in a day at Google’s HTML5 Web App Hackathon event in Chicago, IL.

Known Development Time-sinks

  • Modifying VexFlow data structure and rendering formatter for neumes.
  • Converting neume font to .js and using it within VexFlow.
  • Import/Export MEI

On Canvas versus SVG

Great article describing both the technologies and when it's best to use them.

  • Use SVG: "graphics model persists" versus Canvas "fire and forget". Use when you need to observe/print whole image, but also need to "drill into the detail". When you want to "include an interactive experience involving hover effects, selection, tabbing between items, and scaling." versus Canvas, which "requires you code your own hit detection using the event object’s mouse coordinates"-("Code can and does exist in the form of graphing libraries to enable specific hit detection on graphs using pixel data").
  • Use Canvas: when you require speed but "don’t require retention of what was drawn". If you need to manipulate individual pixels.

Staffless Neume Progress

Here's an update on the progress of Neon.js for staffless neumes:

  1. I've updated all of the libraries that Neon.js depends on. The most major change was the newest version of Fabric.js, the HTML canvas drawing library. Many functions and object properties had changed since the version that the Neon.js was using, which was hacked on quickly by the Fabric.js developer to include functionality for grouping drawings on the canvas. Neon.js now uses a more stable release of Fabric.js.

  2. Keyboard shortcuts.

  3. Glyph opacity changing. This was a feature that was requested every time I presented Neon.js.

  4. After scouring google, I could not find a true type font for staffless neumes like those in the St. Gallen manuscript. Therefore, I couldn't extract SVG drawings from a font, like I did for the Liber Usualis, to display on the canvas in Neon.js. From the list of St. Gallen neumes I received from Timothy, I created custom SVG drawings in Adobe Illustrator for many of the neumes. These neume drawings are in a file in the Neon.js repository and can be displayed on the canvas.

  5. I've refactored all of the Neon.js source code to use "prototype chaining", which is JavaScript's version of inheritance, to allow code reuse across multiple document types. For example, there exists a neume class at the highest level, which stores basic information such as the bounding box size and position. Pitched neumes inherit from this class and add pitch information to the neume and provides functions to derive the neume name based on the pitches of notes within the neume. Staffless neumes also inherit from this class and provide alternate functions. The same structure exists for other musical elements (e.g., staves, clefs, etc). This structure allows Neon.js to be "easily" (notice the scare quotes … it'll still be a huge job!) extended to edit common Western music notation, should that be the goal in the future.

  6. Interface functions such as neumify, ungroup, and head shape change now work for staffless neumes. Since pitch can not be used to derive the neume name in the case of staffless neumes, the vertical position of puncta on the drawing surface determines the melodic movement of notes. Therefore, to draw a Clivis neume, two puncta are inserted with the second lower than the first, and the neumify button is pressed.

While working on displaying and editing the multitude of liquescent neumes in Neon.js, it became obvious that I'm at a point in development where the representation of these neumes in the MEI document needs to be formalized. Looking at the table of St. Gallen neumes, it seems—at least from my understanding—that liquescence is more complicated in staffless neume notation than square-note notation, as there is liquescence of augmented and diminished neumes. The MEI document needs to store this information. As well, Andrew requested that neumes be paired with the corresponding syllables of chant in the MEI document.

Clone this wiki locally