So I watched the new GoT episode last night, and boy was it awesome. While the first episode of the season was more of a recap, and used to get everything back into gear, this one really got into the pace of things. We didn’t get to see much of what’s going on north of the wall, but what we did see established why the wildlings are all united under Mance Rayder, and really gave a sense of how scared they are of the White Walkers. In the North, we finally got to meet Ramsay Snow (god I hate that bastard). Looks like Theon is in for a fun season. I really like it that they decided to move the story of all the things that happen to him to season 3, instead of having him remember later. Hopefully we’ll get to see him transforming slowly (and painfully). I’m also really glad that they finally introduced Jojen and Meera Reed, although they did come a bit out of the blue. Had they been here last season this would have made a bit more sense, but oh well… To be honest, the Bran story line has always been the one that interests me the least (to the point of skipping some of his chapters and going back to read them later), but Jojen and Meera are the ones that have kept it somewhat interesting, so it’s really good they’re here now.

In the riverlands, we continue with the bastardization of Robb’s story line. The main gist of it is the same: he’s a good person and a piss poor decision maker, but the beauty of the books is in the details, and those have been somewhat lost. At least they finally found a way to leave Bolton at Harrenhal. I’m a bit disappointed that we don’t get to see Catelyn by her father’s side as he dies. That would have made for some really good character development. In lieu of that, they did that piece about Jon Snow getting pox which, in all honesty, sent shivers down my spine. All things considered, a very good scene, but unfortunately deviating from the books a little bit (at least as far as I remember). At the same time, we finally meet Thoros of Myr. I really like they way he looks on TV. Looking forward for his interaction with Clegane. And speaking of the Hound, really looking forward to see where his storyline with Arya goes.

But the real meat of the episode, at least for me, came from the scenes at King’s Landing. At last house Tyrell shows its colours (at least to the viewers). We already knew that Margaery wants to be the queen, but now we finally meet the Queen of Thorns and find out Margaery has the full strength of house Tyrell behind her. That scene where both of the Tyrell ladies press Sansa into spilling the beans was incredibly powerful. Particularly the rolling of the eyes Margaery does as she learns Joff is “a monster”. Instead of being scared, she assumes a “I’ll have to deal with that” attitude, and then puts that into action the next time we see her. Not only does she manage to diffuse Joffrey’s mood, but also plays the slightly clueless dame who doesn’t know much about politics (lies! damned lies!), and then approaches His Grace both physically and psychologically. By the end, Joff is convinced she’d like to shoot that crossbow into someone as much as he would. That final image in the mirror is almost frightening. At that point, Margaery is in control, Joff is totally lost in is (blood) lust, and they both look like evil itself. It remains to be seen if Margaery really is playing a part or if that was her shedding a bit of the shell she’s always wearing (ironic on account that she doesn’t wear much else, like Cersei so accurately pointed out).

The only thing I missed was a bit of Daenerys. I really want to see her story move forward. Looks like next week we’ll finally see her getting her Unsullied, which should be the turning point for things in Essos. Still, overall a really good episode with lots of new characters and very interesting story development. This show keeps on doing something really impressive: makes me want to see the next episode right after the one I just saw, despite the fact that I already know how the story.

In any case, I’m glad I read what I found. Thinking I was being an idiot at the time and not finding that sort of idiocy in myself now is a reason to celebrate. Besides, I can always do with some criticism, even my own.

Like I said, I’m using Scala for this course. This was the offending code:

class ControlListener extends Runnable {
    def run {
        val socket = DBS.sockets("control")_1 //acquire socket from central singleton
        var packet = new DatagramPacket(new Array[Byte](1024), 1024)
        while (true) {
            socket receive packet
            spawn {
                process(packet)
            }
        }
    }
    
    def process(packet: DatagramPacket) { 
        val message = packet getData
        //more stuff
    }
}

What I’m trying to do here is pretty simple. I have a class which repeatedly tries to read from an UDP socket, and pass the packet which has been read into an handling function (process). I want to do the handling asynchronously, so I call the process function within a spawn block, which is more or less the equivalent of doing this in Java:

new Thread() {
    public void run() {
        //call process
    }
}.start()

In Java you actually have to call process differently, and spawn does more than this, but the idea is what I’ve shown.

Now, this seems like it should work, but the code was inexplicably blocking on the val message = packet getData line. Further more, it would continue execution when a new packet came in. Cue a lot of head scratching.

Here’s the problem: when spawn is invoked, it starts a new thread which will indeed run process, however, it also goes back to the top of the loop. What happens there? receive gets called again, and it locks access to the packet. Although this makes sense (you don’t want to read from a packet while it’s being written to, and vice versa), I could not find any documentation about this. Now, because this is Scala/Java, things are passed by reference. Thus, the packet within process is the same packet into which the socket is reading. Because access to the packet is locked, process blocks until receive returns, which happens when the next packet comes in.

The fix is obviously very simple: get the data you need from the packet before you spawn the new thread and go back to the top of the loop. Despite this, it took me a whole day to debug this. Here’s the corrected code:

class ControlListener extends Runnable {
    def run {
        val socket = DBS.sockets("control")_1 //acquire socket from central singleton
        var packet = new DatagramPacket(new Array[Byte](1024), 1024)
        while (true) {
            socket receive packet
            val message = new String(packet getData, 0, packet getLength)
            val address = packet.getAddress.getHostAddress
            spawn {
                process(message, address)
            }
        }
    }

    def process(message: String, address: String) {
        val fields = message.trim.split(" ")
        //more stuff here
    }

}

It actually hurts to look at how simple this is now.

If you want to try this on your own, here’s a handy Java class demonstrating the effect:

import java.net.DatagramSocket;
import java.net.DatagramPacket;
import java.net.InetAddress;
import java.net.SocketException;
import java.net.UnknownHostException;
import java.io.IOException;
import java.lang.Thread;

class SocketTest {
	public void test() {
		
	}
    public static void main(String[] args) throws IOException {
        DatagramSocket ds = new DatagramSocket(50000, InetAddress.getByName("localhost"));
        final DatagramPacket dp = new DatagramPacket(new byte[1024], 1024); //packet used to read

        //thread used to try to access the packet's data
        new Thread() {
            public void run() {
            	//give the rest of the code enough time to do stuff
                try { sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); }
                
                System.out.println("Will try to call getData on dp");
                dp.getData(); //should block
                System.out.println("getData ran");
            }
        }.start();

        //thread used to send a packet to the reading socket so that it returns
        new Thread() {
            public void run() {
            	//give everything else time to run and then leave them wating
                try { sleep(5000); } catch (InterruptedException e1) { e1.printStackTrace(); }

                //used to send something to ourselves
                DatagramSocket dsout = null;
				try { dsout = new DatagramSocket(50000); } catch (SocketException e) { e.printStackTrace(); }
                
				//used to carry some data for us
				DatagramPacket dpout = null;
				try {
					dpout = new DatagramPacket(new byte[1024], 1024, InetAddress.getByName("localhost"), 50000);
				} catch (UnknownHostException e) { e.printStackTrace(); } 
                try { dsout.send(dpout); } catch (IOException e) { e.printStackTrace(); }
            }
        }.start();

        ds.receive(dp); //will block until the second thread runs
    }
}

If you run it, you should see “Will try to call getData on dp” come up after about a second, and then “getData ran” after another four seconds (the time left in the second thread, which forces receive to return).
If you do not get these results, please let me know!

After that, things got a bit less familiar. I finally had to learn how to properly configure apache. By that I do not mean that it’s configured as it should right now, as I’ve still got some work to do on it, but I finally learned how to do virtual hosts and all of that. Fun stuff. Then I decided to install something I haven’t had before: phpmyadmin. I really like to use mysql on the command line (and actually did for the rest of the install process), but I figured that it can’t hurt, and it’s pretty handy at times.

And then, I had the brilliant idea of setting up my own mail server, which took me till about 15 minutes ago to get done. Let’s put this way: I am not the smartest guy on the planet, but I’ve installed a good amount of software and configured a couple of different servers, and this was stupidly hard to get done. My final setup is postfix+postfixadmin+dovecot+roundcube, all of them storing information in a couple of mysql databases. I’m not going to get into the nitty gritty, but overall let’s say postfixadmin was very easy, and roundcube wasn’t hard. The real problem was with postfix and dovecot. They’re both VERY sensitive to the configurations, and while getting one to work wasn’t a total nightmare, getting them talking was painful. Alas, I did get all of it working, and just sent an email from the VPS to my gamil account and it worked!

The next step is to get the memory usage down. Right now I’m all out of the 512Mb ram I get, and I still need to get gitosis up and running. This means I might have to change from nginx to apache. That’ll be another fun little adventure, I bet…

Now, surpass hasn’t been particularly amazing over the last couple of years. Their cPanel installation has a couple of bugs, their ssh access is kind of weird, the support is lack luster at best, and often down right useless. We’ve lived with it though, on account that it’s pretty cheap, and most times the thing actually runs and serves web pages.

These last few days have changed everything. The story begins with an e-mail from a couple of weeks ago warning about a server migration starting on the night of the 19th to the 20th of March. At the time, they told us we should expect about 12-14 hours of downtime (that alone for a server migration is “wtf” worthy, but ok). We just ground our teeth, sighed, and accepted that for about half a day there’d be no server. As predicted, my blog disappeared on the 20th at midnight (GMT, 8pm EDT). At the time I didn’t even associate that to the migration (I’d forgotten, in all honesty), but chalked it up to some DNS hickup or something, and on account that I was going to bed, I just let it be.

The next morning I had classes, so I was off the internet for a while. In the afternoon, though, my father told me there were some errors going on. I tried to access the server and everything was either down, or miss configured and throwing errors everywhere. Now getting an HTTP 500 error when trying to access your website’s homepage will scare you, specially if you haven’t been messing with .htaccess and know for a fact that there was no error in configuration last night. Trying to FTP in to see what’s going on and getting your connection denied will sound the rest of the alarms.

Long story short, we contacted support and the answer we got back was along the lines of “Try flushing your DNS cache, that might be the problem”. Now, I know that I thought it could be a DNS hickup the previous night, but at the time I didn’t remember that the server was being migrated. The support staff, on the other hand, had to be aware of that. And yet they replied with the standard “flush your DNS” response. No comments.

Skipping forward about a dozen messages back and forth with support, it’s now the 21st of March. It’s been 30 hours since the thing died. It’s mostly back up, but we’re still getting errors sometimes. Trying to access my web mail threw a HTTP 500 in my face, and I had to use an alternate access. Even after I got in, Round cube is throwing up a message about not being configured, so I’m forced to use horde. Uploading things trough FTP is also still raising some issues every now and then.

To add insult to injury, I decided to look around for reviews of surpass, just to see if anyone else has felt like I do right now. This lovely thing came up: http://hostjury.com/blog/view/55/surpass-hosting-employees-post-fake-reviews. Just…wow!

Because of all of this I’m now looking for new solutions to host my website(s). One of the options I’m considering is Linode. Has anyone used it? Is it any good?

</rant>

PS: I realize it’s ironic that the fact you can read this is proof that the server is up right now, as this blog is still hosted on surpass for now.

Here’s what I came up with: the browser runs, with the help of jQuery, 3 ajax requests: one to connect, one to write messages and one to get messages. This last one is launched, waits for the server, and then is re-launched as soon as the server replies with something (this is known as long polling). On the other side, the server offers 3 matching services: connect, writeMessage and getMessage, via 3 PHP scripts.

Everything is tied together by a simple (literally a form to write stuff and a div where messages show up) html+css interface on the user’s side, and by a file to store the latest message on the server side.

So, without further ado, let’s get into some code. First up, the server side.

connect.php

<?php
$chatId = 1; //for now
echo json_encode(array("chatId" => $chatId));
?>

Yes, that is ridiculous. This is only here because the idea is that we want a series of parallel chats, and the clients get to know which one they’ll end up in upon connecting. Obviously not implemented at this stage.

Moving along, the getMessage.php code:

<?php

if(empty($_GET['chatId'])) {
	exit();
}

$filename = dirname(__FILE__)."/../chats/".$_GET['chatId'].".txt";
$lastmodif = empty($_GET['timestamp']) ? 0 : $_GET['timestamp'];
$currentmodif = filemtime($filename);

while($currentmodif <= $lastmodif) {
	usleep(10000);
	clearstatcache();
	$currentmodif = filemtime($filename);
}

$response = array();
$response['message'] = file_get_contents($filename);
$response['timestamp'] = $currentmodif;
echo json_encode($response);

?>

Now it gets interesting. What the server is doing is taking a request with the time the client last read a message, then sitting in a loop waiting for the file containing the messages to be updated. Once it is, it sends the contents back to the client. This is obviously not very efficient on the server side, but that was not the point of this prototype.

Finally, here’s writeMessage.php

<?php

if(empty($_POST['message']) || empty($_POST['chatId'])) {
	echo json_encode(array("status" => "ERROR: no data", "data" => print_r($_POST)));
	exit();
}

$filename = dirname(__FILE__)."/../chats/".$_POST['chatId'].".txt";
file_put_contents($filename, $_POST['message']);

echo json_encode(array("status" => "OK"));

?>

All the writeMessage service does it receive some data and replace the contents of the chat file with that data. Yes, this does mean you can spam the server and effectively prevent anyone else from getting their messages saved long enough for people to read them. Again, not the point.

And that’s it for the server (aside from some configurations, such as turning magic quotes off)

On the client side, all we have is a chat.js Javascript file. I’ll break it up into functions to make it easier. Fair warning: this makes extensive use of jQuery. If you have any doubt about what one of the functions does, check out http://api.jquery.com/.

chat.js:connect

function connect() {
    $.ajax({
        type : "GET",
        url : "services/connect.php",
        async : true,
        cache : false,

        success : function(data) {
            var contents = $.parseJSON(data);
            chatId = contents['chatId'];
            //setTimeout('waitForMsg()', 500); <-- bad way of doing this
            setTimeout(waitForMsg, 500); //do this instead
        },

        error : function(XMLHttpRequest, textStatus, errorThrown) {
            console.log("Unable to connect: " + textStatus + "(" + errorThrown + ")");
        }
    });
}

So this is the pair to connect.php and all it does is get the chatId from the server and stick it into the correct variable. This is obviously meant to be expanded.

chat.js:waitForMsg

function waitForMsg() {
    $.ajax({
        type : "GET",
        url : "services/getMessage.php?timestamp=" + timestamp + "&chatId=" + chatId,
        async : true,
        cache : false,

        success : function(data) {
            var contents = $.parseJSON(data);
            $("#messageBox").append(contents['message'] + "<br />");
            timestamp = contents['timestamp'];
            setTimeout(waitForMsg, 500);
        },

        error : function(XMLHttpRequest, textStatus, errorThrown) {
            console.log("Error reading: " + textStatus + "(" + errorThrown + ")");
            setTimeout(waitForMsg, 10000);
        }
    });
}

This is the function that handles asking the server for new messages. It does an ajax request which is then left hanging until the server replies, at which time the success function runs, and adds the message that’s been received to the DOM. The request includes the time stamp (which is then updated when the server replies), as well as the id of the current chat. At the end, the function is scheduled to run half a second later, so the polling process keeps on going.

chat.js:loadSendHandler

function loadSendHandler() {
    $("#messageForm").submit(function() {

        $.ajax({
            type : "POST",
            url : "services/writeMessage.php",
            data : {
                "chatId" : chatId,
                "message" : "<b>" + $("#messageName").val() + ":</b> " + $("#messageInput").val()
            },
            async : true,
            cache : false,

            success : function(data) {
                var contents = $.parseJSON(data);
                if (contents['status'] === "OK") {
                    console.log("server said OK");
                } else {
                    console.log("server error'd: " + contents);
                }
                $("#messageInput").val("");
            },
            error : function(XMLHttpRequest, textStatus, errorThrown) {
                console.log("Error writing: " + textStatus + "(" + errorThrown + ")");
            }
        });
        return false;
    });
}

This function loads an handler for the submit event on the form used to type in messages. All it does is send the data typed in, along with the name chosen by the user, to the writeMessage service on the server side.

And finally, here’s what runs when the DOM is done loading (and a couple of global variables declarations for good measure).

chat.js

var timestamp = null;
var chatId = null;

$(document).ready(function() {
    connect();
    loadSendHandler();
});

Like said in the beggining, this is all tied together by a little html and a little css.
index.html

<!DOCTYPE html>
<html>
    <head>
        <title>Livechat</title>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="style.css">
        <script src="//ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
        <script src="scripts/chat.js"></script>
    </head>
    <body>
        <div id="wrapper">
            <div id="messageBox">
                Messages will show up here
                <br />
            </div>
            <div id="formBox">
                <form id="messageForm">
                    <input type="text" name="name" placeholder="Type your name here" id="messageName" />
                    <br />
                    <input type="text" name="message" placeholder="Type your message here" id="messageInput" />
                    <br />
                    <input type="submit" name="Send" value="Send" />
                </form>
            </div>
        </div>
    </body>
</html>

style.css

#wrapper {
    width: 800px;
    margin: auto;
}

#messageBox {
    margin: 5px;
    min-width: 500px;
	border: 1px solid black;
	float: left;
}

#formBox {
    margin: 5px;
    width: auto;
	border: 1px solid black;
	float: right;
}

Just a little css so we get a fluid centered page. To be honest I hardly put any effort into this, on account that I was very near my two hour limit.

This little experiment was educational in two ways. Number one, I learned some new client <-> server with jQuery <-> PHP tricks. Number two, and most important, I learned this kind of thing is really easy to prototype fast. I might grab some more quick little projects like this in the future.

Finally, let me face the elephant in the room. The code above is obviously riddled with security holes. To be honest, I think it’s more like the whole thing is one giant security nightmare. But keep in mind that this was a prototype. An experiment. One of the things I want to do now is fix all of the problems I can find, and see how long it takes and how much bigger the code gets. Please, do not, I repeat DO NOT use this code. It kind of made me nervous to have it up just to test it, so yeah…

We were counting on the Club’s long-standing policy of offering their services to the direct family of a member, as if they were members themselves. Well, as it turns out that’s no longer the case, and they wanted to charge me and my mother one year’s membership to take care of the matter, on top of their service fee and the service fee the state charges. On the other hand, if we use the public service, all we have to pay is said state fee. On top of that, the process is simpler with the public service than it would be with the Club, with less documents going back and forward and whatnot.

So it seems that a private entity is both charging more and being more of a PITA than the state. This in a country where public services are famously bad. What the heck?!

People tend to say blogs are like an on-line diary. They are, as long as you publish said diary in the newspaper (which, by the way, goes against the definition of diary). The truth is, blogs are a different type of writing medium. I don’t pretend to know much (anything, really) about blogging. I’ve only dabbled in it occasionally and for short periods. And yet, I think I can conclude this: if there’s even one person you’d rather not read it, don’t publish it.

Back to bed.

Scala being Scala has access to all of Java’s standard API. However, getting RMI to work is actually not straight forward, so here goes a short guide, on account that I couldn’t find a condensed source of information anywhere. We’ll implement a simple echo service in 4 steps

1) RMI requires you to create an interface extending the java.rmi.Remote interface. This will be the general contract of your system, which both the client and the server must be aware of. In Scala, you can do something similar to interfaces using traits (traits are actually more powerful, but for the purposes of this post let’s assume they’re the same). Here’s what a trait might look like

import java.rmi.Remote
import java.rmi.RemoteException

trait Echoer extends Remote {
    @throws(classOf[RemoteException])
    def echo(input: String): String
}

The @throws(classOf[RemoteException]) annotation is necessary because all of the methods of an interface used for RMI must be capable of throwing a java.rmi.RemoteException. In java you would state this by adding throws RemoteException after the method declaration. The annotation is Scala’s way of doing the same.

2) With the trait created, you now need to create a class which implements it. It has to extend the trait, implementing the method(s) we’ve declared there. It also has to start the service and bind it to the rmi registry. Enough talk, here’s some code:

import java.rmi.server.UnicastRemoteObject
import java.rmi.registry.Registry
import java.rmi.registry.LocateRegistry

class Server extends Echoer {
    def echo(input: String): String = {
        return input //just return what we got
    }
}

object Server {
    def main(args: Array[String]): Unit = {

        try {
            val server : Echoer = new Server
            val stub =UnicastRemoteObject.exportObject(server,0).asInstanceOf[Echoer]
            val registry = LocateRegistry.getRegistry()
            //the bit between quotes can be anything
            registry.rebind("echoservice", stub)

            println("Server ready")
        }
        catch {
            case e: Exception => e printStackTrace
        }

    }
}

We’re taking advantage of Scala’s class <-> companion object duality here. Note that these have to be defined in the same file. In case you’re not familiar with Scala’s companion objects, there’s a good explanation of what they are here.

The class itself simply implements the method. If this were a real application it’d be much larger, of course. The object, on the other hand, is doing a lot of stuff. The server’s main method is embedded here. This is where we get the rmi registry, create an instance of our service, and then bind it to a service name. Once this is done, the application will run until we stop it.

3) The last bit of code needed is the client. We’ll implement a very simple client which sends a message to the server and prints out what it gets back. Before it can do that, however, it has to fetch the stub for the server from the rmi registry.

import java.rmi.registry.Registry
import java.rmi.registry.LocateRegistry

object Client {

    def main(args: Array[String]): Unit = {

        try {
            //replace localhost with the server's hostname or IP address
            val registry = LocateRegistry getRegistry("localhost") 
            
            //the name looked up has to match the
            //name the service is bound to in the server
            val stub = registry.lookup("echoservice").asInstanceOf[Echoer]
            
            println("Sending \"Hello world!\"")
            var response = stub echo("Hello world!")
            println("Got \""+response+"\"")
        } catch {
            case e: Exception => e printStackTrace
        }
    }
}

As you can see, the client also has to know about the Echoer trait.

4) With everything coded up, you need to start the rmi registry. This is done by running start rmiregistry on windows (if it asks about allowing it trough the firewall, just say yes to everything) or rmiregistry & on linux. If you’re told that rmiregistry doesn’t exist, make sure you have the path to the bin/ folder of your JRE/JDK installation added to your PATH.

5) Finally you can launch your application. If you’re using eclipse like I am, you’ll need to create a run configuration for the client and another one for the server. If you’re launching trough the command line, just run the proper scala command. Either way, you need to add this to your VM options, both for server and client:

-Djava.rmi.server.codebase=file:path/to/class/files/

The trailing slash is important! In eclipse’s particular case, the same argument would be something like:

-Djava.rmi.server.codebase=file:${workspace_loc}/ProjectNameHere/bin/

And voilà! RMI on Scala without writing a line of Java.

PS: My thanks to Pedro Cunha and Tiago Vila Verde for helping me getting this thing working.

Anyway, I get to the university this morning, get an espresso, and sit outside with my laptop drinking it and reading e-mail and other assorted stuff (by assorted stuff I mean I opened Reddit and looked at titles that sparked my interest). As I’m doing this, I find out that Google reader is closing. This doesn’t affect me in any way, except that people are going to be asking for an alternative RSS reader for a couple of weeks. I never really understood RSS, or the need to have your favourite websites’ content delivered to you in a condensed, lossy way when you can just go to the website. But all that aside, that got me thinking about the repercussions of a product so many people use just closing down. To be honest, my first thought was “maybe people will stop using RSS now”. Which got me thinking about blogs, which led me to mine and wanting to reboot it.

Well, I got that done, but then…I thought about the concept of reboot, and how there are some posts here of which I am not particularly proud (mysql ids, I’m looking at you). I honestly considered just deleting everything and starting over. But here’s the thing: I simply can’t. Most of you know this, and I’m pretty much preaching to the choir, but some people need to learn a basic fact of life (hi, records’ industry et al). You can’t remove anything from the internet. Period. If you “delete” your facebook, it’s still there. If you take some photos down, someone saved them and will eventually post them again. Etc, etc, etc. Once it’s on the internet, it’s there forever. Yes I can remove the data from my website, but there’s always, if nothing else, Google’s cache to keep it for you. Which brings us back to them. Google’s everywhere these days. Can you imagine if they were to just shut down? I think we might actually loose some data then…

PS: on the topic of unique mysql ids, I have to address that one of these days. But for now, suffice it to say I was being an idiot. One word: transactions.