Monday, October 15, 2018

AWS Lambdas.. use context or callback to respond to the caller?


AWS Lambdas are great. No need to run your own servers and all that, simply awesome. However working with Lambdas introduces Lambda specific problems and responding to the caller is one of those things you just need to get familiar with if you need to do anything more than "hello world".

Ok, so when AWS Lambda is invoked it's handler will be executed. With runtime version before 4.3.2 the way to respond to caller was by using context.succeed(), context.fail() or context.done() e.g.:

exports.handler = function(event, context) {
    // something here
    context.succeed();
}

This context call was resulting in lambda being "frozen" at this very point, even if Node's event loop was not empty. This could be the case when some asynchronous operation wasn't finished, connection to database was not closed etc. Now, as this lambda invocation was not finished and so new lambda invocation might (not necessarily though) use the same container, the result of leftovers in event loop might apparently be quite unpredictable.

With version 4.3.2 third parameter to handler function was introduced and so handler might look something as below:

exports.handler = function(event, context, callback) {
    // something here
}

This new callback parameter is an improvement on previous context.succeed(), context.fail() and context.done(). It takes err or success parameters and more on callback(err, success) itself you can find in AWS docs.

Unlike calling context.succeed/fail/done, calling callback(..) will not immediately freeze lambda execution, but rather wait till the event loop is empty. It solves an issue with non-empty event loop when lambda being frozen, however it also results in confusing behaviour as many expects lambda to immediately finish and this is not the case.

To cater for this behavior callbackWaitsForEmptyEventLoop property was made available on context object. Setting it to false e.g.:

exports.handler = function(event, context, callback) {
    context.callbackWaitsForEmptyEventLoop = false;
}

will result in Lambda execution freezing immediately on callback(..) call and leaving content of event loop as it is.

Even though I'm aware of the risk, I found this usefull when connecting to database and it helped me to reuse connection pool and avoid closing opening database connections with every lambda invocation.

Tuesday, October 7, 2014

Do you Promise?


This post is a summary of a short presentation I gave a while ago at work. Its is a brief introduction to Promises in JavaScript. It doesn't explain the exact syntax or what Promise object is but rather what I consider as important advantage of using Promises over old callback approach and why every JS programmer should get familiar with this design pattern.

If you code JavaScript, sooner or later you will have to deal with asynchronous operations and callbacks. These callbacks as we know have its problems. You as a programmer often end up with code like that:

someAsyncOperation(function(data) {
    // do stuff
    // or something
    anotherAsyncOperation(function(data2) {
        // do stuff
        // or something
        andYetAnoterAsyncOperation(function(data3) {
            // do stuff
            // or something
            imRunningOutOfNamesAsyncOp(function(data4) {
                // do stuff
                thisSucksAsyncOp(function(data5) {
                    // do something
                    // or who cares now?
                });
            });
        });
    });
});


Pyramid of doom, callback hell, call it what you will. This is far from elegant solution and even though that above example is nicely indented and has hardly any actual code in it, it is already hard to read. Add some logic to it and you got yourself a mess. Trying to maintain it will be a pain and room for errors is enormous.

As ugly as it is, it’s not the main problem however. Look at the code below and try to guess what will be its output:

try
{
    setTimeout(function() {
        throw new Error(“BOOM”);
    }, 2000);
} catch (ex)
{
    console.log(ex);
}

This illustrates main problem with vanilla JS callback approach. The Error thrown inside of callback will not be caught inside of catch block. It will fail miserably. So will this code:

someAsyncOperation(function(data) {
    // do stuff
    // or something
    anotherAsyncOperation(function(data2) {
        // do stuff
        // or something
        throw new Error(“DOH!”);
        andYetAnoterAsyncOperation(function(data3) {
            // do stuff
            // or something
            imRunningOutOfNamesAsyncOp(function(data4) {
                // do stuff
                thisSucksAsyncOp(function(data5) {
                    // do something
                    // or who cares now?
                });
            });
        });
    });
});


Imagine this happening in your NodeJs code. It will very likely bring your process down. You can of course use domains or AFAIK depreceated process.on(“uncaughtException..) but there is a better way.

That's where Promises come in. It’s a design pattern that allows you to deal with callback hell and, most importantly, deal with exceptions. Imagine that instead of the mess above you can write code like this:

someAsyncOperation()
.then(function(data) {
    // do stuff
    return anotherAsyncOperation();
})
.then(function(data) {
    // do stuff
    return andYetAnoterAsyncOperation();
})
.then(function(data) {
    // do stuff
    return imRunningOutOfNamesAsyncOp();
})
.then(function(data) {
    // do stuff
    return thisSucksAsyncOp();
})
.catch(function(err) {
    console.log(err);
    return err;
});

Callback hell is no more and structure looks much more flatten and readable. Instead of nested callbacks, each .then(..) is executed only when previous one (returning a Promise) finishes. Top to bottom, very much like synchronous code. Also notice the catch block at the end of the chain. Readability is important of course, but the main point of using promises is the way it let us deal with exceptions. Exceptions will be caught in catch block and will let us recover gracefully. Without doubt this is most important point this design pattern delivers.

Now, this very simple example in no way covers everything you need to know about promises. I didn't even explain what the Promise object is, .then method and other details. I hope however it illustrates important point, that Promises allow us to deal with exceptions in an elegant way. This alone should interest you in Promises enough to google and experiment further. There are tons of articles available on the net. Google and try as its important to see for yourself how promises simplify development. If you are interested also have a look at my short presentation that is available here.

Thursday, September 25, 2014

My struggle with JSON

We all use developer tools. Console is of tremendous help when it comes to trying JavaScript snippets. Sometimes however something relatively simple e.g. like typing in JSON throws errors for apparently no reason. I had one of these moments and as usual started digging to see why it was happening. Ok, hit F12 in Chrome and type in:

{ “name” : “homer” }

Plain, old JSON it seems, nothing fancy. However you will be greeted with:



It is valid JSON so what gives? First thing that came to my mind was hidden characters. It’s (not so) well know fact that JSON is not quite a subset of JavaScript. JSON allows certain characters in a String and JavaScript doesn't. Given that one of the strings above contain Line Separator (line feed, carriage return etc.) this would sure spell trouble. Even though “Unexpected token :” would point problem to “:” rather than anything else I decided to check strings for any whitespace characters.

Fired up my trusted Notepad++ to see what Chrome doesn't show, but with no luck. No hidden characters of any type.

To make the matter worse running:

{ name:”homer”}

worked just fine. Almost fine actually, as the line was returning a “string” rather than object:



Why string though? Let me explain.. Using curly braces in console the way I originally did, creates a block statement. It is not parsed in expression context and so it does not create an Object. It does very much the same as I would execute:



It produces exactly same output so error pointing to “:”. If you look at this however it suddenly stops making sense. We simply creating here a string, followed by a colon, followed by another string. Clearly wrong.

Now, { name:”homer”} however was working. It was returning string rather than object, and for a good reason. Curly braces create block statement and so lets strip those and run:



You should see the problem by now. Obviously what I was trying to do here is to create a label followed by string “homer” and so “homer” was spit out to console as expected. If you are not familiar with labels in JS they are used e.g. to break out of loops e.g.:

var i, j;

loop1:
    for (i = 0; i < 3; i++) { //The first for statement is labeled "loop1"
        loop2: for (j = 0; j < 3; j++) { //The second for statement is labeled "loop2"
            if (i == 1 && j == 1) {
                continue loop1;
            }
            console.log("i = " + i + ", j = " + j);
        }
    }

// Output is:
// "i = 0, j = 0"
// "i = 0, j = 1"
// "i = 0, j = 2"
// "i = 1, j = 0"
// "i = 2, j = 0"
// "i = 2, j = 1"
// "i = 2, j = 2"
// Notice how it skips both "i = 1, j = 1" and "i = 1, j = 2"


So there it is. I was creating block statement with invalid syntax hence the error. Mistake that I won’t do again.

Tuesday, September 9, 2014

Help children and get healthy! its almost too easy!

I thought I should share it everywhere as it is for a good cause, helping children..  I encourage everyone to participate in Steptember, its easy and makes a big difference in lives of children and their families. At least have a look what Steptember is all about and participate if you can!

Monday, September 1, 2014

Memoize like a boss (part 2)

Memoization is fun and if you read my previous post you should have fair idea of how memoization can be easily implemented in JavaScript. Previous examples were using very simple function and now you might wonder how to deal with recursion and still get all the benefits. Last time around we end up with:

Function.prototype.memoized = function(a) {
if (typeof this.cache === "undefined") this.cache = [];
if (this.cache[a]) {
return this.cache[a];
} else {
this.cache[a] = this(a);
return this.cache[a];
}
}

Function.prototype.memoize = function() {
var t = this;
return function() {
return t.memoized.apply(t, arguments);
}
}

myTest = (function superLongCalculation(someArg) {
    return someArg * 3;
}).memoize();

console.log(myTest(2));


So to recap.. myTest function once called will call memoized method on original function object t:

t.memoized.apply(t, arguments);

memoized method will do the check if the value for argument ("2" in our example) has already been precalculated and if so will returned whatever is stored in cache. Otherwise will call original function, do the magic and return new value. If this doesn't quite make sense have a look at my original post where I went into details of my basic memoization implementation.

So now you might wonder what will happen if instead of something overly simple as above we will use recursive function to do the work. Calculating fibonacci numbers seems to be "Hello World" of memoization and so lets plug this thing into our memoization code:

Function.prototype.memoized = function(a) {
if (typeof this.cache === "undefined") this.cache = [];
if (this.cache[a]) {
return this.cache[a];
} else {
this.cache[a] = this(a);
return this.cache[a];
}
}

Function.prototype.memoize = function() {
var t = this;
return function() {
return t.memoized.apply(t, arguments);
}
}

myTest = (function fibonacci(n) {
return n < 2 ? n : fibonacci(n - 1) + fibonacci(n - 2);
}).memoize();

console.log(myTest(2));

Works like a charm. Returns correct values, however we can make it better. It still does unnecessary work and given our current code there is easy way to improve it.

Notice how the fibonacci function object itself is memoized, however internal calls to itself are not. This means that in this case we do half-assed optimization only and anything half-assed is not cool. Lucky for us the Function object has memoized method already that we can directly call as in the very first examples in part one:

Function.prototype.memoized = function(a) {
if (typeof this.cache === "undefined") this.cache = [];
if (this.cache[a]) {
return this.cache[a];
} else {
this.cache[a] = this(a);
return this.cache[a];
}
}

Function.prototype.memoize = function() {
var t = this;
return function() {
return t.memoized.apply(t, arguments); }

}

myTest = (function fibonacci(n) {
return n < 2 ? n : fibonacci.memoized(n - 1) + fibonacci.memoized(n - 2);
}).memoize();

console.log(myTest(2));

I also put few tests that are available here so you can see the difference. I also realise that there are other, more efficient ways of calculating fibonacci numbers, I believe however that examples above are simple enough to see how memoization can be used with recursive functions.

Tuesday, July 8, 2014

Memoize like a boss!

Concept of memoization is not new. There is like a billion articles on the topic. Out of many, Resig covered it in his Secrets of JS Ninja book and Stefanov in his JavaScript Patterns. Addy Osmani blogged about performance of different memoization techniques and if you've been leaving under a rock and you are not familiar with it google is your firiend. There is a lot of good stuff out there so make sure you check it out. Long story short memoization is a method of caching the results of a function, you know, to make it faster.

Imagine that superLongCalculation() function takes a parameter and returns value of some sort.

function superLongCalculation(someArg)
{
  // some complicated stuff is happening here with someArg provided and only when done return result

  return result;
}

Now, wouldn't it be nice if the function, after doing all the work could remember what the result is for this particular argument? Next time when its called with the same argument it should be able to look it up and return it right away, without recalculating same thing again. This is fairly simple to implement, just look at my attempt in JS below:

function superLongCalculation(someArg) {
    return someArg * 3;
}

Function.prototype.memoized = function(a) {

    if (typeof this.cache === "undefined") this.cache = [];
    
    if (this.cache[a]) {
        return this.cache[a];
    } else {
        this.cache[a] = this(a);
        return this.cache[a];
    }
}

console.log(superLongCalculation.memoized(2));

For the sake of example I keep it simple. Calculation is not all that complicated and code is pretty much self explanatory. We simply extending Function object with additional method. This memoized method, once called with argument, will check internal array if there is a precalculated value for the argument we passed. If so then will return this precalculated value. Otherwise will call original function (this(a)), this function will do very long calculations and the new value will be returned. To call it we simply use:

superLongCalculation.memoized(2)

Easy stuff yet very powerful. Yes, I know, extending native JS prototypes is a bad idea in general blah blah blah.. IMO it is useful in certain situation and as long as you understand what you doing and possible problems that come with it I see no issue.

Ok, then I stumbled upon Secrets of JS Ninja by Resig. He further extended the method allowing memoization to be called directly on a function. Real brainfuck if you ask me. And very interesting one. It allows retrieving cached results directly out of a function:

  superLongCalculation(2)

To implement this we need to go a bit further however. The code I managed to put together is below and explanation of bits and pieces will follow:

Function.prototype.memoized = function(a) {
  
    if (typeof this.cache === "undefined")  this.cache = [];
    if (this.cache[a]) {
        return this.cache[a];
    } else {
        this.cache[a] = this(a);
        return this.cache[a];
    }
}

Function.prototype.memoize=function() {
  var t=this;
  return function() {
   return t.memoized.apply(t,arguments);
  }
}

myTest = (function superLongCalculation(someArg) {
    return someArg * 3;
}).memoize();

console.log(myTest(2));

memoized function stays as it is. As previously it takes parameter and checks cache for precalculated values.

So lets start with figuring out what each bit is doing:

myTest = (function superLongCalculation(someArg) {
    return someArg * 3;
}).memoize();

We simply creating a function here:

(function superLongCalculation(someArg) {
    return someArg * 3;
})

It takes argument, multiplies by 3 and returns. This is in fact Function object and as such has memoize method we attached to Function prototype:

Function.prototype.memoize=function() {
  var t=this;
  return function() {
   return t.memoized.apply(t,arguments);
  }
}

Further we have a call to memoize method:

.memoize();

This call will give us back another method that will do stuff later. So to sum it up from now on, when we call myTest, in fact we will call:

function() {
   return t.memoized.apply(t,arguments);
};

This method has access to function object we created earlier (t) and in turn calls memoized method on this very object. Everyone's following? Right, so now at some point in our program we get to call myTest and spit out result to console:

console.log(myTest(2));

So now myTest(2) calls:

function() {
   return t.memoized.apply(t,arguments);
}

t (via closure) refers to function object we created earlier. On this object we call memoized method with "2" passed as parameter. memoized method checks if value is in cache, if so returns it, otherwise calls original method and then returns the result. And so we have basic memoization implemented now.

Now, that was the easy part and it gets a bit more involving when you try to make superLongCalculations recursive. If anyone actually read this post I might put together few more examples with fibonacci numbers or something later, for today its enough however.

Tuesday, June 24, 2014

AngularJs custom directive trouble

So I decided to build very simple directive in AngularJs. Came up with this simple code:

<!doctype html>
<html>
  <head>
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.0/angular.js"></script>

    <script>

var app = angular.module('myApp', []);

      app.directive("taWithMessage",function() {
    return {
    template: "<p><h2>sample</h2></p>",
    replace: true
    };
   });

    </script>

  </head>
  <body>

    <div ng-app="myApp">

<div ta-with-message></div>

    </div>

  </body>
</html>

.. ran it and BOOM! Got this:


To make it more interesting replacing:

template: "<p><h2>sample</h2></p>",

with

template: "<p><span>sample</span></p>",

or:

template: "<div><h2>sample</h2></div>",

Worked just fine. It obviously didn't have anything to do with display of the element as div s well as p are both block, yet div works fine, p didn't. 

Then I started debugging and after a while I realised what a dummy I've been. Given original code, the browser was interpreting it as:

<p></p><h2>sample</h2><p></p>

This excerpt from specs should clarify what the issue was:

A p element's end tag may be omitted if the p element is immediately followed by an address, article, aside, blockquote, div, dl, fieldset, footer, form, h1, h2, h3, h4, h5, h6, header, hgroup, hr, main, nav,ol, p, pre, section, table, or ul, element, or if there is no more content in the parent element and the parent element is not an a element.

If you ever read the HTML specs you already know that some elements do not need to have closing end tags. I was well aware of this yet at this point didn't realised that was the issue. 

Decided to dig further into AngularJs source itself to see what exactly is going on in there and as it turned out innerHTML is the culprit:



AngularJs nicely passes template code to JQLite (highlighted in yellow below) and JQLite at some point runs it via innerHTML. This spits out modified as per above specs code. Then as shown below, AngularJs checks if the code is valid, so have one root only:



One root meaning:

<div><p>something</p><p>something else</p></div>

Rather than:

<p>something</p><p>something else</p>

As the code JQLite spits out is:

<p></p><h2>sample</h2><p></p>

It fails the check and Angular throws Error.

All this Googling and head scratching could be avoided if I only remembered the basic specs. Shame on me.. =/