Dev Guide
From Gemin-Wiki
| Table of contents |
|
|
Introduction
This document should explain roughly how Gemin-iPlus works, and give a developer a route into the code, perhaps suggest ways that a coder might make a simple change or two.
Assumptions
We assume that:
- you have manged to install G+ on you machine. If you do not have a running G+ install, look at the Install_Guide.
- You have a basic knowledge of Java programming
- You have a basic knowledge of Flash/Actionscript programming
- You have a basic knowledge of Unix Adminsitration (running command line scripts etc.)
Overview
Gemin-i Plus has three essential elements:
The Flash Client
The client is compiled from the .fla file (and the #included .as actionscript files) by loading the .fla into Flash MX and selecting "Export Movie". This will produce a .swf file which needs to be copied into the webroot of your server so that it can be downloaded by a user and connect back to the server when required
The .fla file is in the client directory named gemin-iplus.fla
Almost all of the actionscript code itself is in the .as files in the client/scripts directory. This enables us to edit those files through our CVS system, keeping past revisions and allowing code-merges if two people are working on the same code at the same time. Please try to keep this convention if you edit the code, keeping actionscript in included txt files rather than in the .fla binary.
Important Note: Editing the .fla
IMPORTANT: Please see our note about editing the .fla files if you intend to use CVS to commit any changes you make. The .fla is a binary file and can not be merged if two people make changes. You should always LOCK the .fla before you make changes to it, and check that nobody else has it locked before you do so.
The PHP/Apache Admin Scripts
The .swf file described above will be copied into the doc-root of your local web server. In that doc-root with the .swf file are a few PHP scripts which do basic admin tasks like editing user details etc. These scripts also handle user uploads since until Flash9 this was impossible under the flash security model.
The php files are in apacheroot/gemin-iplus along with the rest of the files which need to be in the webroot somewhere. Note, though, that these files are copied into the web-root at install-time. So altering the version in apacheroot/gemin-iplus won't alter the version which is used by your browser when testing. If you're editing these files you'll have to edit the version in apacheroot/gemin-iplus (so that the alterations end up in CVS) and then copy the file into your webroot at the appropriate place before you test.
The Java Server
The server runs a JAVA servelet which listens on a port (default:8080) for http connections containing XML requests from the Flash Client. The requests usually just ask the server to interrogate the database (sometimes making changes to it) and return an XML document reflecting the success/failure of that operation and/or a XML object describing some thing the Client should show (a homepage for example).
The server's source files are in the src/net/gemin-iplus directory.
The Java source files are compiled into Java by running ant from the main gemin-iplus directory. The buildfile.xml there tells javac where to find the source and where to put the .class files.
Run the server by running ./gemin-d from that same directory.
The flash movie
Of course you will need Flash installed to be able to edit or compile the flash movie, at least version seven, ideally the latest version of Flash.
The movie is architecturally quite simple, just 3 frames in the main stage with the majority of the complexity being in the large collection of objects in the library.
Frames
The .fla file consists of three 'frames' on the stage. The first is the initialization frame, the second is the main 'running' frame and the third is the 'fatal error' frame.
Initialization
When the flash movie executes it starts on frame 1 (init). This #includes "preload.as" which in turns includes just about all the various actionscript .as file libraries that the whole application needs, and initializes all the global state variables. After that, it loads some config files over http from the web server which the flash file came from. Especially "client_config.xml"
Search for "client_config.xml" in the preload.as file to see that configuration loading code. It basically checks each XML node and sets the values of those global variables mentioned earlier for each recognised type of node.
If there's a problem with any of that, it calls _root.fail() which switches to frame 3 (the error frame) and shows the client log.
Assuming there was no problem, it sends a request to the java server asking for a sessionID, and calls gotoAndPlay("run") which flips over to frame 2 (Running).
Running
Frame two includes "mainactions.as" which creates the menus and desktop etc, sets up a ticker to periodically ping the server. This means that the "checkReqeuestInterval()" function (defined in preloader.as) is called every few seconds to make the client check with the server to see if any new actions have come in (new mail, chat requests, whatever).
Finally, it adds the "Login" window. The login window is an object in the object library which we'll talk about more shortly.
Fail
The fail frame is small and easy, it just consists of a textbox which contains the client log and some code to make play stop.
The Object Library
If the object library isn't visible, select "Window/Library" to display it.
The objects in the library are divided into 3 main folders:
Flash Components
"Flash components" contains standard flash objects like buttons and scrollbars and checkboxes. You shouldn't need to edit these.
Gemin-i Plus Components
This folder contains other, custom, icons and controls. These are mostly links to external .swf libraries which can be altered (by renaming the .swfs on the server) to change the look and feel, the skin. If you click the "properties" of most of these objects you'll be able to see the link to the gui/defaultSkin.swf file where the actual look of those objects is defined.
Some of the objects aren't skinable, and these are all defined directly in the library.
Classes
We have classes to define the functionality of many of these components, too many to list here, but we will examine a few of the most important:
Window Manager Class
Contains functions like getNewWindow() which creates a new Gemin-i Plus window, sets up it's default size and other properties, adds it to the component stack, adds an entry to the taskbar, and returns a handle for the whole window.
It also has lock/unlockAllWindows() which is used to implement modal dialogs, where no other windows may be changed until some question has been answered.
It implements the window focus, and the 'expose' mode, allowing all windows to be viewed at once for quick switching between windows and
Window Class
Most things which can appear on the Gemin-i Plus screen are kept inside windows, which are controlled by the window Class. The window class handles the edges of a window, it's title and size and borders, the 'history' of each window, allowing the "Back" button to be implemented, as well as controlling the 'toolbar' at the top of most windows. Each window has a container in the middle, which contains a renderer for some other object. We will talk about renderers in the next section.
Tabstack Class
client/scripts/tabs.as contains the TabStack class. This allows a window to show various tabs, each of which contains a different renderer to display different things inside the window. For example, almost all windows contains a 'view' tab which shows the content and a 'viewers' tab to show the permissions for the page.
Here's some code, taken from the User Preferences renderer, which shows how a tabstack is added to a window.
var tabsPanel=this.createEmptyMovieClip("tabsPanel",this.getNextHighestDepth());
tabsPanel.container_width = container.container_width;
var tabstack = this.tabsPanel.attachMovie( "renderer_tabStack", "renderer", this.getNextHighestDepth() );
var generalTab = tabstack.getNewTab("General") ;
generalTab.renderer = generalTab.pane.attachRenderer("preferences_general", {id:id});
generalTab.onShow = function(){}
This code attaches a new tabsPanel to 'this' (IE the prefs renderer), ensures it has the same width as 'this' window, adds a stack to that panel, and creates a 'general' tab inside that stack to which it attaches a renderer "preferences_general"
Renderers
Every object that can be displayed on the Gemin-i Plus screen has a "renderer" object. Obviously these are defined in the library by the objects in the "renderers" folder.
Most of these simply #include a .as file which will contain code to attach() various components and position them, though some still use the old and deprecated method of having had the components actually dragged into the object in Flash. We prefer to use code to attach() the components and move them becasue editing them doesn't mean locking the .fla, thus more than one person can work on the code at once.
All the renderers have a "render" function and an "update" function, both of which are called, in that oder, when a new renderer is first created. The "render" function attaches and arranges components within the container and is called once at setup time. The 'update' function is called each time the window is resized, and will likely need to rearrange those components but is unlikely to need to add new ones.
We won't list *all* the renderers here, there are far too many, but we will describe some of the more important ones in further detail.
ObjectLoader Renderer
Most of our renderers are in files names client/scripts/Renderer-X.as, but some were created before this naming convention and don't have that Renderer_ prefix. objectloader is one of these, in client/scripts/objectloader.as
Typically, when a window is opened in Gemin-i Plus, it's renderer is set to an objectLoader renderer. This displays a loading indicator, usually a rotating circle of some kind, and makes a call to the server to fetch the contents of the object it's asked to render.
When it gets a reply to that request, the function handleResponse() is called. That has some error handling, showing the "Permission Denied" page if the object wasn't allowed to be loaded for exxample. If there wasn't a problem it examines the type of theobject, and if it's one of a handful of special types, it changes it's own container to attach a special renderer matching that type.
Otherwise it calls displayObject() on it's own container class, which replaces the whole renderer with the objectView renderer.
ObjectView Renderer
client/scripts/ObjectView.as is also a renderer which is older than the naming convention. In fact, this one is so old it doesn't even have a render and update function, just a displayObject() function which, well, displays the object.
It sets up a tab-stack, sets one tab to be the 'view' of the object. Depending on object type, it may also set another to show the 'viewers', IE the relations, permissions and owners of the object, and another to show the 'members' of groups or 'memberships' of users, or 'Info' for objects that can be tagged.
Finally, in the "view" tab it attaches a renderer based on which type of object is may be. EG: a page would get a 'page' renderer, an image would get the 'image' renderer.
Page Renderer
The page renderer is one of the more complex of all our renderers. It shows pages, which are XML documents describing where various textboxes, circles, arrows, and embedded-objects are laid out.
It's render() function attaches some buttons to the window depending on the type of page. For exmple, buttons to vote to 'recommend' or 'report' a page.
Next it attaches a scroll-pane to the content area. This allows the user to scroll around the page even if the window is smaller than the page content.
If the page is editable, it attaches a switch to allow the user to switch to edit mode. It sets up a stack of cells and then calls the parsePageStructure() function.
This in turn goes through each element in the XML representing the page. If it finds an embedded object, it creates a cell in the appropriate size and sets that cell's renderer to the "loadObject" renderer.
Client/Server communication
Requests from the client are sent over HTTP to Jetty, usually listening on port 8080. Jetty in turn passes the request to the Gemin-i Plus server. The requests are XML documents. Here's a typical example of a request:
<request>
<session>1198088142331_7hndof</session>
<call id="1">
<function>getNotifications</function>
<argument type="text"><name>userID</name><value>11034</value></argument>
</call>
<call id="2">
<function>getUsersBans</function>
<argument type="text"><name>userID</name><value>11034</value></argument>
</call>
</request>
Note that each request will contain a session ID, which the server uses to track which users are making which requests, as well as zero or more "calls". A call causes the server to execute a function, which will likely add a return to the XML reply sent back to the client.
In this example, the client is making two calls at once, one asking to see if any new notifications have been added since the last call (including chat invites, notifications of buddy's logging in, gem-mail messages etc.), and one asking if any new bans have been added since the last call. This request is certainly the most common one, each client sending a request like this every few seconds.
Another common XML request looks like this:
<request>
<session>1197994441133_Lmxrfw</session>
<call id="1">
<function>getObject</function>
<argument type="text"><name>id</name><value>11034</value></argument>
</call>
</request>
Which is, fairly obviously, a request to fetch the content of object number 11034.
The details of exactly how these requests are passed from client to server are beyond the scope of this document, but we should point out an example in the source-code of how to use the libraries to build a request on the client side, pass it to the server, processes it there, and send the result back to the client.
A login request
As an example, we'll look at the code that handles the login request.
Building The Request
The file client/scripts/Renderer-LoginB.as contains the following code: (NB: The "We'll deal with this bit later" comment there is actually a few dozen lines of code which handle the reply. We'll get to that later.)
// make a login request
var request = new GemRequest();
var call = request.getNewCall("attemptLogin");
call.addArgument("username", this.usernameField.text);
call.addArgument("password", this.passwordField.text);
call.addArgument("mic", _root.hasMicrophone().toString());
call.addArgument("cam", _root.hasCamera().toString());
if(_root.schoolsFolderID != undefined){
call.addArgument("schoolsFolder", _root.schoolsFolderID);
}
call.onResponse = function() {
//We'll deal with this bit later
}
call.onFailure = function() {
// handle failure
_root.log("GemRequest failed!");
};
request.submit();
The first line creates a request, and the next creates a call within that request. After that, four lines add arguments to the call, and the next 3 optionally add a fifth. The next 3 set up a function to call when the reply is recieved, and the 4 ofter that st up a function to call if there's an error. Finally, the last line submits the request.
Note that this code returns before the reply is received. Indeed, the libraries may not even send the request immediately. All we have to do is build that XML, set the callback functions and call the submit() function to send it.
Processing the request
The request bounces around inside Jetty and the Gemin-i Plus Server's libraries for a while, being stripped down to individual calls and eventually each call ends up in src/net/geminiplus/flash/FlashClientManager.java at the "execute" function.
This is the hub where (most) requests get sent off to the relevant function to process them.
The majority of the function is a large array of if...else which calls a single function if a call with that functions name appears. The bit relevant to our example is this fragment:
else if( call.getFunctionName().equals( "attemptLogin" ) ) {
funs_attemptLogin(call,session);
}
which passes the session and the call to the function which actually does the login code.
Sending a reply
That function, also in src/net/geminiplus/flash/FlashClientManager.java, extracts the values of the arguments from the call object, and does things like check the database to see if this user supplied the right password and username, that they're not already logged in etc. We're skipping over those bits and concentrating on the communications architecture at the moment.
Replies to the client are also XML documents, and the "call" object itself supplies us with the method to pass back values. The attemptLogin function sends a surprisingly large amount of information back to the client. It sends global object numbers for special objects like the folder containing new-page templates and the group identifying "all students" etc., the users school_id, whether the user is an administrator. Lots of things. But they each have a line in the code similar to this:
call.addReturn("success", "yes", GemParameter.TEXT );
call.addReturn("firstName", newIdentity.getFirstName(), GemParameter.TEXT );
call.addReturn("lastName", newIdentity.getLastName(), GemParameter.TEXT );
as you can see, we're returning name/value pairs, and each value has a "TYPE", usually just TEXT.
After building this reply for the client, funs_attemptLogin returns back to execute, which in turn returns back up the stack of library functions to eventually send Jetty the XML reply for the whole request, containing replies for each call in that request.
Finally, Jetty sends a reply to the HTTP request, the body of which contains exactly that XML.
Client receives the reply
Remember when we built the request we skipped over a part of the code saying we'd come back to it later? This is where that code comes in. When we built the request we added on "onResponse" to it, which was a function. In the case of the login request that function was build with a lambda (unnamed function) in-line.
Here's the code in client/scripts/loginB.as which we replaced with that 'deal with this later' comment earlier. There are lots of fragments along these lines:
if (this.getReturn("goldMemberGroup")) {
_root.goldUsersIcon.info.objectID = this.getReturn("goldMemberGroup");
trace("Set Gold Group ID to "+_root.goldUsersIcon.info.objectID);
}
Which fairly transparently simply store the infrmation passed. "this", is the call itself and "getReturn()" is the function which returns the value of a name/value pair when passed it's name. In this case the number identifying the group listing all gold members is just stored in a global state variable.
It also has these lines:
var success = call.getReturn("success");
if (success == "yes") {
...
]
which trap a successful log in and then perform the complicated actions of adding the users data to the client's list of logged-in users and launching that users home-page.
And that's it. We've traced the most common path of communication from client to sever and back. Hopefully you followed along in your own copy of the source.
The java server's structure
The java server can be compiled by running Ant from the base directory.
It is a Java Servelet which is contained within jetty, a java web-server. All the source files are in src/net/gemin-iplus/
Here's a list of a few of the major classes and what they handle...
Kernel.java
This is the top-most class in the server. It contains lots of state variables, and the init code, plus the doGet and doPost methods which actually grab the information in the HTTP requests passed from jetty. It also contains the logging functions which are used to dump information to the debug logs. The code which reads kernel_config.xml to set up the system configuration is in Kernel.java, and it also sets up all the other classes and, where appropriate, instantiates them with global objects.
kernel_config.xml
The first thing the kernel does is to find and load kernel_config.xml - this XML file allows a sysadmin to change some things about the setup of the system. Which port to listen on, what messages to allow to the log, what to call the logfile, what the hostname of the system is, where backups should go, where ffmpeg can be found to convert video files, how the code is distributed across a cluster of servers, where to find the database. Things like that.
You can find kernel_config.xml in jetty_root/config/
It's generated at install-time from a template, kernel_config.xml.ORIG, and can be edited by the sysadmin at any time, though the server will need to be restarted to make it reflect changes.
Logging
The kernel_config.xml file allows the sysadmin to set a 'logging level' between zero and 100 to fine-tune how quickly the logs fill up with trivia. Setting to to, for example 80, means that only messages with a level 80 or higher will be added to the log.
Each call to kernel.log should pass both a message to put into the log-file AND a log-level. Errors should have high levels, 80 to 100. Warning should be between 50 and 80. Debugging trace-statements should have a log-level of 50 or less.
The logging systems also allows us to define logging sections. This code defines the sections:
public int logsection_startup = 1; public int logsection_shutdown = 2; public int logsection_ticker = 3; public int logsection_auth = 4; public int logsection_messaging = 5; public int logsection_objdb = 6; public int logsection_servercalls = 7; public int logsection_archive = 8;
A typical call to the logging system should be like this:
kernel.log("Message here",kernel.logsection_auth,50);
though often messages end up without a section for one reason or another, notibly all the messages added before the section system was there.
Obviously being able to log trace-calls is important for debugging.
Database.java
Database.java is a small wrapper around the mysql libraries. For historical reasons stretching back further than my time on the project, it keeps a bank of query results. These results don't need to be freed, so the application can make SQL queries, view the results, and just let it go at that.
Database.java does things like attempt a reconnection of the DB connection fails. It also keeps track of TWO database connections, one for reading and one for writing. This helps when running on a distributed system: writes go to the master, reads come from a slave.
Example SQL call
Here is some sample code showing how to do a database query. It just fetches and writes to the log the name of the first ten objects. Database code must always be wrapped up in "try" blocks to catch exceptions:
try {
String queryString = "SELECT id,title from obj_objects where id<10";
ResultSet resultSet = database.query( queryString );
while(resultSet.next()){
String name = resultSet.getString("title");
int id = resultSet.getInt("id");
kernel.log("Item "+id+" is called "+name,50);
}
}catch(Exception e ) {
kernel.log("Exception in test code"+e.toString());
}
Constants
There are three files to merely define some global static values.
Definitions.java
Definitions.java contains things like the ID for the "Registered Users" group and the "Everybody" group and the "Site Homepage" etc.
Types.java
Types.java contains the meanings for the "Type" of various objects. For example it tells you that objects of type 401 is a content page. You'll understand this more whe nyou read about the obj_objects database table.
Privs.java
Privs.java contains the meanings of links. For example, if an object is linked via the obj_links table with a link type '1001' then we know that this object is the PUBLIC AVATAR for that object. You'll understand this more when you read about the obj_privs databse table.
flash/FlashClientManager.java
The Flash Client Manager is designed to manage requests from a flash client. We have already mentioned that the backbone of this class is the execute() function which is passed a gemCall and a session. The gemCall gives it the function to handle the request and the parameters to that function while the session gives information about which user is making the request etc.
Most of the functions that the Flash Client can call are defined within this class too, simple database queries to fetch or set information.
Some functions, however, will be calls which are expected to manipulate or alter objects, or use server plugins (like the Chat function) and these generally call functions in LocalObjectManager.
LocalObjectManager.java
This class looks after local objects. It has functions to getting the members of a group, fetching lists of administrators, tracking owners of objects, finding which priv level a user has over an object (can they read? write?), things like that.
It's also home to some of the biggest ugly hacks in G+. "Special Pages" are pages who's ID is less than zero. Negative numbers. There's functions in this class to generate the XML which describes a page that lets a user enquire about gold membership, or list all the available projects, or show and pick from the avatar gallery.
LocalSessionManager.java
The session manager keeps the database up to date with knowledge about which connections have which users logged in. Each sessions represents a connection from a client flash application to the server. Each may have zero to five users logged into that session, since we allow more than one user to log into a scren at once.
Notable functions in the LocalSessionManager include
getSessions() - return a hashtable of all sessions currently active logout() = log a user out of a session getSessionFromUser() = find which session a user is logged into attemptLogin() = log a user into a session removeOldSessions() = delete sessions that haven't updated in TIMEOUT seconds getNewSession() = make a new session, initially with nobody logged in attemptChangePassword() = change a users password
The Database
Intro
We won't discuss all the database's tables. Mostly things like ids_bans are self-evident. That one contains the list of bans, for instance. Here are the most important of the database tables however:
obj_objects Table
Every object in Gemin-i Plus has an ID number, and an entry in the obj_objects table. Which means every objects has an ID, a type, a timestamp, a title, some content and a meta-field.
The java file src/net/gemin-iplus/Types.java defines what the types mean.
Some objects, images for example, have filenames in the content field rather than the content itself.
Gemin-i Plus users have attached objects, and loading that object into a window shows that users homepage.
obj_privs Table
The obj_privs table tells us which users own which objects. For example, if object 100 is created by the user who's object_id is 10, then a link would exist in the obj_privs table saying (parent=10, child=100, type=31).
The definitions of the different types of 'ownership', from "Administrator" to "Viewer" is in the java file privs.java.
Note that a single object may have many attached privs, that users AND groups can have privs over an object, and that a single user will have privs over many objects.
obj_links Table
The obj_links table tells us where an object is, and an object may be in more than once place at a time. for example, a folder may have object ID 50, and an image in that folder may have object ID 100. In this case there'd be a row in the obj_links table saying (parent=50, child=100, type=1).
A definition of what different types of link mean is available in the java file Links.java
Note that a single object may be linked to many things. For example, a message may be in someone's inbox, and if it was sent to two peopoe, in someone else's inbox too. And that user may have put a link into a folder to that object to. And of course, a folder/inbox/group may have many members too.
obj_history Table
Because many users can edit a page, pages are open or vandalism. So we keep a version history of every page. This table tracks that version history, allowing the users to go back to version 1 of their homepage, say, if the page is vandalised.
obj_threads Table
The Gemin-i Plus forums are fully threaded. Forum posts show replies in a tree format. This table tracks which objects are related in this tree. We use a "Modified Preorder Tree Traversal" system. See <a href="http://www.sitepoint.com/article/hierarchical-data-database/2">this page</a> for more information.
There are functions to add, delete etc. from threads in the LocalObjectManager class.
sm_sessions_objects Table
The sessions table tells us which sessions are currently active: IE they have a client using this session code to connect. If no entry exists with a given session code, we can assume nobody is logged in at that session. If someone is logged in, their object_id and the timestamp they last contacted the server and the status of their camera and microphone are recorded here.
Deleting all sessions effectively logs everybody off immediately.
ids_identites table
The identities table holds each users login information. Note that the ID in the ids_identies table is NOT the same as the users object ID. The ids_identity_objects table links the ids_identities ID with the object ID for that user.
Passwords, usernames, country, admin status, last-login time, and a list of groups for that user are in this table. It's not really accessed much outside login code.
API
The set of functions defined in Gemin-i Plus for the general use of applications buit to run on it are varied, and still mostly undocumented. We will try to keep up documentation on new ones, and as we edit old ones, on the Gemin-iPlusAPI page though.
Beware though, that page is very incomplete. If you expect there's a function for doing something, look at the source to find it, that'll be a much better way than relying on the documentation.
SiD
SiD (Schools Information Database) is a separate application, written in Ruby, which can access the same database as Gemin-i Plus. It's used as a Customer Relationship Manager, allowing our staff and volunteers to keep track of Gemin-i's relationship with schools. Registering a new school to the system is a process which happens under SiD, as is tracking payments and which members are staff. SiD has a database entry for all the schools we have contact with, not just those who have Rafi.ki accounts.
SiD allows our administrators to disable and/or delete accounts, change passwords, track communication with schools, specify projects that schools can join and many other things.
![[Main Page]](/wiki/stylesheets/images/wiki.png)