Slice bug not fixed – delete session

Slice has a neat feature – it can save a session in an XML file, and that can be loaded later. I’ve been planning to use that for examining the annotations students make.

The problem was that the load session feature didn’t work so well. It was giving me different errors on different machines, including a C# exception on one of them.

All the errors had to do with networking, so my original plan was to disable networking completely. When I realised it – I thought it was unfortunate that pretty much everything revolves around networking (original design/requirement), but as I was digging into it I realised that the saved session XML is in the same format as the UI XML. Then I tried to load the session on startup, which worked.

So I’m not fixing that bug, since I have a way around it.

Slice bug fixed – delete line numbers/code

I think I mentioned before a serious bug in Slice – that the user can use the eraser tool to delete not just the annotations but the line numbers and the code itself. When Mike and Zuzel played with my customised Slice – Zuzel managed to delete the code 3 times. The result of a manifestation of this bug is that the annotations are still there, but the code isn’t – so the user has to start over.

This must have been very frustrating for her, and it was scary for me, my experiment will have been ruined if it happened to more than one or two participants.

Thankfully this wasn’t very hard to fix. With Sam‘s help I set up a development environment for working on the Slice core, which gave me some debugging ability, and a 3-line changed fixed the bug. Now I’m going to challenge everyone I can think of to try and delete the code 🙂

Setting up ReviewBoard

Last week I’ve spent a couple of days setting up reviewboard. It wasn’t as painless a process as I had hoped, but in the end it worked so I’m happy.

Partly the problem with installing it is that I’ve never installed a python-based website before. The closest I ever got to setting up a Web2.0 website was setting up my blogs (this being one of them). But WordPress is written in PHP (which Slackware has support for out of the box) and they’ve put a lot of effort into making the installation as dummy-proof as possible.

I downloaded the ReviewBoard .egg file as the instructions suggested, but apparently that wasn’t needed at all. I also downloaded and installed setuptools, which is a sort of a python package manager that automatically without prompting downloads python modules (or apps?) that are dependencies for what I asked for.

The easy_install app from setuptools isn’t very hard, but it’s scary. It downloaded and installed a dozen things without asking me a thing, stuff like Django which I’d really want to know where it’s installed and for what purpose. I guess if I really wanted to understand what was being done I should have done a manual install, but given my lack of knowledge in the field and my lack of spare time – that would have taken weeks.

Luckily I haven’t done this on (my production server) but on a secondary, disposable server. This one is on a DSL like so its upload is pretty poor (50KB/s) but it should work for the experiment. It may even be a good thing that it’s slower than normal since my tablet is pretty old too (at times it’s obvious when you scroll quickly).

The next challenge was the prerequisites that the easy_install didn’t handle. One of them is a choice between mod_fastcgi and mod_python. Slackware didn’t have either installed. I tried fastcgi first guessing it would be easier, but it wasn’t – I couldn’t even figure out what it’s supposed to do, so it was hard to guess how to set it up. So I installed mod_python instead. A manual install, but wasn’t hard.

I brushed up on my MySQL admin knowledge and set up a database and a user with the right permissions. I wish I didn’t have to google this every time. The MySQL manual is really good, but it lacks the first time barebone setup I do once or twice a year.

While messing with the above I had to run rb-site install a couple of times. The second time it didn’t work and I had to delete all the tables it created the first time. There is no command in MySQL to dump all the tables from a database, but I found a handy script that I used successfully.

The most difficult part of the process was the Apache setup. even not counting the failed fastcgi installation attempts it took quite a few tries to get it right. The final solution (not ideal, but it works) was to create /etc/httpd/extra/httpd-vhosts.conf included from httpd.conf with these contents:

NameVirtualHost *:80
<VirtualHost *:80>
DocumentRoot “/home/www/htdocs/rb/htdocs”

# Error handlers
ErrorDocument 500 /errordocs/500.html

# Serve django pages
<Location “/”>
PythonPath “[‘/home/www/htdocs/rb/conf’] + sys.path”
SetEnv DJANGO_SETTINGS_MODULE reviewboard.settings
SetEnv PYTHON_EGG_CACHE “/home/www/htdocs/rb/tmp/egg_cache”
SetHandler mod_python
PythonHandler django.core.handlers.modpython
PythonAutoReload Off
PythonDebug Off
# Used to run multiple mod_python sites in the same apache
PythonInterpreter reviewboard_rb

# Serve static media without running it through mod_python
# (overrides the above)
<Location “/media”>
SetHandler None
<Location “/errordocs”>
SetHandler None

<Directory “/home/www/htdocs/rb/htdocs”>
AllowOverride All

# Alias static media requests to filesystem
Alias /media “/home/www/htdocs/rb/htdocs/media”
Alias /errordocs “/home/www/htdocs/rb/htdocs/errordocs”

This is almost the same as the sample httpd.conf ReviewBoard created for me, except for the first line. It would have been nice if somewhere in the docs it was mentioned that ReviewBoard requires a dedicated host to run on, but that’s ok – I already mentioned this is a disposable server.

After I finally got to see my ReviewBoard setup in a Firefox and changed the default username/password – there was just the matter of posting a patch into it. Again this was strangely difficult to do. It appears that a ReviewBoard instance is intended to work with one and only one source repository. I’m used to svn and have a server set up so I tried to connect it to that. I found (after googling it) that I need pysvn, so I installed it.

Then I was still confused about how I would go about posting a diff. I stumbled through it, and found one combination of “Base Directory” and “Diff” that worked. For this one I had to do a diff (specifically with LANG=en_CA svn diff -rx:y). That’s fine, I can just commit whatever I want reviewed into my homework repo.

Here’s what it looks like:

Screen recording software

One of the things I’m sure I’m going to need and use is screen recording software, so that I can replay what my participants have done when I do my analysis.

I tried a couple of tools. One of them was ScreenFlash, but that one slows down my machine terribly even if doing a single frame per second. The second one I’ve used before: BBFlashBack. It installs its own video driver so I guess it’s making the screenshots more directly, and saves files in its own format, to be exported as Flash later. It’s not free but works really well, I think I’m going to use it.

The app in the recording is Slice, the one I’m going to use for the tablet portion of the experiment.


Later I also have to see if I can find some working way to record participants reviewing things on paper too.

Finally, found hardware

Finally I found a tablet PC to use in my study. After a lot of searching, and checking out many of those listed on Craig’s List I got an HP/Compaq TC1100. It’s amazing how many people want real money for real junk.

It’s a relatively old machine, with a 1GHz PentiumM, a half a gig of ram and a 40GB drive, but a nice 12” screen and a very well working stylus. The seller claimed it was practicaly unused which I didn’t believe at first, but now I do – there’s hardly more than one or two scratches on it, and no damage of any other kind. That’s a big deal, you’d know why if you ever tried to buy a used laptop.

I thought it looked familiar, and yes indeed: Prof Plimmer whom I keep mentioning used the same at least once:

When assembled as a laptop it looks a little strange, but there’s a reason for that – the screen not only flips like most tablets, but detaches completely from the keyboard, made my girlfriend say it’s like an iPad. I’ll leave that comparison for another day.

And it came with a nice carrying case:

Interestingly even together with the case it’s smaller than both my HP g60 and my work laptop 🙂

I tried to upgrade the memory in it, and even after finding the slot (it was doubly hidden) I found that the 1G stick I have is too tall, won’t fit in. And the spare 100GB drive I had lying around turned out to be a SATA drive. So I guess I’ll keep my own monster, only use this one for the stufy, and then sell it. That would be too bad, it’s a really nice machine. I’m even willing to believe that the seller paid over 2000$ for it new.

Ethics Review Board approval

The UofT ERB approved my application, so I’m now officially allowed to run my experiments. All that stands in my way now is some software customizations, lack of a tablet, and lack of volunteers.

I’ve already started playing with the Slice sourcecode. It’s a bit scary but I’m sure I’ll figure it out, especially with Sam’s help.

I’m going to go on Craig’s List to see what tablets I can find, and what I could sell my laptop for.

Volunteers I’ll wait on finding until the other two problems are solved.

Code review using tablets – literature review (4)

I’m doing a hell of a lot of literature review. I’ve heard it said that one needs to stop this process at some point, or else one is doomed to never finish a single paper. Well.. I’m going to finish reading the papers I printed out (they seemed the most relevant stuff available at one point) and later I’ll branch off a little into code review (to date I’ve concentrated almost entirely on tablets). But this has got to end soon 🙂

The guys who wrote “An architecture for ink annotations on web documents” did something with the DOM, somehow combining ink annotations and the rest of a webpage. But the paper is really vague, and mentions overlays and absolute positions. So I’m not sure how (if at all) it is relevant to my work.

“Developing marking support within Eclipse” is completely irrelevant to me, it’s purely about marking.

“Ad-hoc Collaborative Document Annotation on a Tablet PC” is mostly about the implementation details of a distributed annotation system. But they have a screenshot in the end which looks very much like Slice (granted, they all look like Paint).

“Annotation: from paper books to the digital library” talks about annotations on used textbooks. It’s a really interesting read, with an attempt at classifying annotations, but because it’s about textbooks – the classification doesn’t apply to my work. Despite the title digital annotations are barely mentioned at all. The “W3C Annotation Working Group” is mentioned, but it doesn’t seem to exist any more.

“Toward an ecology of hypertext annotation” is also mostly about annotated books and talks about nearly identical issues. Ah, it’s by the same author too.

I mentioned “Moving Markup: Repositioning Freeform Annotations” before. Re-reading it I found an interesting classification of annotations: “explicit selection, textual annotation, and freeform digital ink”. Despite that all three of these are interesting to me in the context of code review the authors didn’t elaborate on the details of each, only freeform annotations (hence the title). The new version of their software (XLibris) was designed to handle collaborative annotation – hence the need to reposition annotations so they look the same on different hardware/software. Out of scope for me.

“Beyond paper: supporting active reading with free form digital ink annotations” is also about XLibris. There are some interesting points in it, but I can’t think of a way to associate them with my work.

“Onscreen marking support for formative assessment” is completely irrelevant to me, don’t know why I mentioned it before.

“Robust annotation positioning in digital documents” isn’t clearly relevant but concentrates on the human side of reflowing annotations, which is almost unique and sometimes interesting. Having other people understand annotations is hard they say, but since they combined their study with reflow – it’s hard for me to judge how it would apply to my work. And curiously in their discussion of existing solutions they mentioned MS Word (I use its Note functionality extensively in collaborative work) basically has the reflow problem solved from the users’ point of view 🙂

“Spatial recognition and grouping of text and graphics” – another one I don’t care about. It’s about the implementation details of turning ink input into geometric shapes.

“iAnnotate: Exploring Multi-User Ink Annotation in Web Browsers” implemented theirs using Silverlight. Mentions that the .NET framework has excellent support for digital ink, though they’d be a bit biased. Mentions “u-Annotate: an application for user-driven freeform digital ink annotation of e-learning content” as prior web annotation work, that one used flash. I’ll try to avoid going down that path (i.e. I have too much literature to review). Anyway – it seems that both these were built for annotating webpages, not even anything structured like homework.

I mentioned “Pen-based interaction techniques for organizing material on an electronic whiteboard” – a paper about electronic whiteboard stuff. It’s way too specific to group meetings, so I didn’t finish reading it.

“Teaching with Tablet PCs” mentions that some school programmes have been set up to experiment with tablets in the classroom (Notre Dame Tablet PC Initiative, SHU Tablet PC Research Project). Said that annotated documents (e.g. slideshows) are useless after the fact, since the annotations only made sense coupled with the speaking (hopefully not the case with reviews which are planned to be stored and reviewed later).

“Experiences with digital pen, keyboard and mouse usability” is a summary of the things Plimmer worked on. An interesting note was “In each case the pen input alone has been insufficient for efficient interaction”.

“Web Page Marker: a Web Browsing Support System based on Marking” – not interesting.

“Preliminary experiences with a tablet PC based system to support active learning in computer science courses” is about student participation in class, not relevant.

“Effects of annotations on student readers and writers” claims that “Studying and Annotating Electronic Text” found that “research has found little difference between annotating digital documents with an electronic pencil and annotating in a traditional paper and pencil condition”, I wonder if that’s really true. If it is – I’ll want to read that for my paper review component. Otherwise the paper is irrelevant.

This tool (Slice) almost does what I want

One of the papers I read (and mentioned in a much earlier lit review post) “A System for Developing Tablet PC Applications for Education” mentioned that it wasn’t built for code review and it wasn’t even fully implemented. Well.. that was published in 2008 which means it was written probably in 2007, long time ago for someone who cares about their software and has time to work on it.

I wrote to the authors and one of them (the lead, Sam Kamin) replied. I’ll give him credit here and now for spending time replying to several emails, it’s very nice of him.

He mentioned that the software is now a framework and it’s called SLICE. Has a website. Is downloadable. Later I found it has a code review mod (they call mods “applications”). Have a look:

This does nearly everything I need. Yes it looks like Paint but I could hardly hope to do much better in the part time I have to implement it. You can open multiple files, scroll through them, handwrite anything anywhere with any colour, erase annotations. It’s actually much more fancy than I need (for example supporting multiple clients working with the same session).

So I could use this with minor modifications. Most likely I would only need to work on the extension and if I need to change the framework itself that option may be made available to me. What’s the problem? It’s that writing code is a mandatory component of my dissertation, and I can only think of some minor changes I would need to make to it:

  • Add some syntax highlighting(maybe)
  • Add the ability to place textboxes anchored to points in the documents (because some things are much easier to type than handwrite)
  • Add a textbox to the global comments window
  • Get rid of the signon dialog (I don’t need the multiple clients feature)
  • Open several files at startup (will be handy when running the study after more than a handful of people)
  • Change the buttons

I have to find out (from my Liverpool dissertation advisor) whether such scholarly uninteresting changes would be acceptable for an outstanding dissertation, or barely good enough. Hopefully the requirement is just to make sure that the graduates are capable of writing code (which I’ll have no trouble demonstrating).

I feel that the meat of the work as planned so far was always the experiment (comparison) and not the code. Will find out soon whether that was wishful thinking.

Project control via self-imposed carrot & stick approach

Some time this week I’m going to have to come up with a project plan for my dissertation. Not only is this a Liverpool requirement but it’s a good idea for myself to make sure I don’t forget to do things that need to be done, and I don’t procrastinate too much.

Coming up with the timelines should be doable (if rough, given the unknowns such as hardware acquisition and participant availability), but I could use some extra mechanism to keep me motivated to do things within my self-imposed interim deadlines.

Thinking of that I just got an idea – what if I create my project plan and then make sure that I lose money whenever I miss some deadline? This could be implemented by giving someone some significant ammount of my own money and have them pay it back to me in chunks, but only given that my work is completed on time?

There’s something to this idea, and I have to seriously consider whether it’s doable.

Scribbling on a tablet

Greg helped me get a hold of a tablet PC (thanks Khai Truong!). It’s small, possibly too small for running the experiment but should be good enough for development of the software I’ll need. So I’ve played with it a little.

It turns out there is a massive problem with one of my assertions. I assumed that one can use a tablet to handwrite, but that’s really difficult, at least on this particular device (a Fujitsu Lifebook P1610). The problem is that the point where the stylus physically touches the screen is 2-3mm off from the point where the computer thinks it’s touching the screen. So I can write one letter just fine, but it’s hard to connect it to the second letter after lifting the stylus:

P1610 Stylus Off

Depending on which way the tablet is turned (you can turn the screen 360 degrees) this can be less of a problem. I was even thinking it might be a good idea to use the thing in full tablet mode with a USB keyboard.

Another thing that may be particular to this device or common to all tablets is that one needs moderate pressure on the stylus, which I didn’t do at first – it seems freaky having to push on an LCD screen with the point of a pen so hard. I pasted a screenshot of some code in notepad into paint, and did some exercises with the stylus. As expected – the offset I mentioned is a serious problem – it’s hard to underline or circle stuff or annotate anything unless you’re doing it on the side.

Also – writing more than a couple of words needs a lot of space (which could be explained by the same dreaded offset which doesn’t allow me to write in small letters). Here’s the masterpiece (I didn’t actually review code, so don’t mind that):

notepad/paint code review screenshot

Overall – there is hope, especially if I manage to get a more accurate/sensitive device.