Tim's Blog

Another code blog

  • RavenDB Survival Tip #2: Simulating select count(*)

    A lot of the time a user will ask for a count of some business entity on a report so they can tell how many of something is happening.  With SQL, this is a very natural process with aggregate operations:

    select count(*) from Users where IsAdmin = true
    

     

    But with RavenDB, I found it difficult to wrap my mind around how you would do this with an Index.  Aggregate operations like this are implemented with map/reduce indexes.  Since the reduce step of the index needs to have the same output shape as the map, we can’t simply return a single numeric value as is the case with SELECT COUNT(*) in SQL.  We need to perform a LINQ group-by.  But in this case, I don’t really want to group by anything in a document, I just want a count. 

    After a little thinking and digging around, I found a solution.  Maybe this is obvious to most, but I found that if we simply group by a constant, we can get a single numeric value for that constant the represents the total count.

    Here’s a sample Index definition that demonstrates this concept.  In the Index, I am trying to find all User documents that qualify as admins (IsAdmin = true).

    public class Users_AdminCounts : AbstractIndexCreationTask<User, Users_AdminCounts.ReduceResult>
    {
        public class ReduceResult
        {
            public string Name { get; set; }
            public int Count { get; set; }
        }
    
        public Users_AdminCounts()
        {
            // the Name parameter to the reduce function is dummy
            // to get it to aggregate to one single value
            Map = users =>
                from user in users
                where user.IsAdmin = true
                select new { Name = "Total", Count = 1 };
    
            Reduce = results =>
                from result in results
                group result by result.Name
                into g
                select new { Name = g.Key, Count = g.Sum(x => x.Count) };
        }
    }
    

     

    When you run the query and get the result (ReduceResult), you will get just one single result and the Count property will contain the count aggregate you are looking for.

    Categories: RavenDB

    Tags: RavenDB, SQL

  • Building a Dynamic Application Menu with Durandal.js, Knockout, and Bootstrap (Pt. 2)

    In the previous post, I laid out a design for creating a dynamic menu system, specifically the object model that will be used in data binding.  In this post, we’ll look at the Knockout bindings and HTML structure to render the menus.

    This will be pretty straightforward as far as taking our object model and applying markup to it, however, the one complicated part is dealing with sub-menus.  Because menu items can themselves contain other menu items, we need the ability to render a menu within a menu and so on.  We can do this with Durandal’s compose binding handler for KO.  It allows recursive composition that is perfect for hierarchical things like menus.

    Here’s the contents of views/menu.html which is the view component for a single Menu rendering:

    <ul class="dropdown-menu" data-bind="foreach: items">
        <!-- ko if: $data.text !== undefined -->
        <li data-bind="css: { disabled: !command.canExecute(), 'dropdown-submenu': hasSubMenu }">
            <!-- ko ifnot: $data.hasSubMenu -->
            <a href="#" tabindex="-1" data-bind="command: command">
                <span data-bind="text: text"></span>
            </a>
            <!-- /ko -->
            <!-- ko if: $data.hasSubMenu -->
            <a tabindex="-1" data-bind="text: text"></a>
                <!-- ko compose: { view: 'views/menu.html', model: $data } -->
                <!-- /ko -->
            <!-- /ko -->
        </li>
        <!-- /ko -->
        <!-- ko if: $data.divider !== undefined -->
        <li class="divider"></li>
        <!-- /ko -->
    </ul>
    

     

    This markup is a bit complicated so let’s go through it:

    1. In Bootstrap, applying the dropdown-menu class to a <ul> will style it as a drop down menu container.  The data-bind here is also set to loop over each item in the Menu object.
    2. Within the <ul>, we need to decide if the MenuItem is a text-based menu item or a divider.  We do this by detecting if the divider property exists and/or there is text to display.  If the MenuItem is a divider, the special divider class is applied to a <li> and Bootstrap renders it as a thin gray line.
    3. If the MenuItem is actually a text-based item, we style it appropriately in its <li> element.  Notice the binding to command.canExecute().  In KoLite, if you provide a canExecute() function on a command (with is a computed observable), it can determine if the command can be executed or not.  In this case, we want the UI to gray-out or disable if the command cannot execute.  Once the command can execute, it will immediately synchronize the UI element to be a clickable command.
    4. Inside the <li>, we check to see if the MenuItem is a sub-menu or not.  If it is not, we create an <a> element as desired by Bootstrap to create the link to click on, binding the appropriate command to it.  We also bind the text in a <span> element inside the <a>.
    5. If the MenuItem does have a sub-menu, we assume that the MenuItem can’t be clickable, but instead, simply groups other elements.  In that case, we call upon the Durandal compose binding handler to recursively call this view sending the MenuItem as the view model to bind to in that context.

    Now, to render the top-level menu bar, in our main view we’d add the following markup (I’m using nav-pills in Bootstrap to represent the top-level menu items, but you don’t need to do that):

    <ul class="nav nav-pills menu-bar" data-bind="foreach: menus">
        <li class="dropdown">
            <a class="dropdown-toggle" href="#" data-toggle="dropdown" role="button" data-bind="text: text"></a>
            <!-- ko compose: { view: 'views/menu.html', model: $data } -->
            <!-- /ko -->
        </li>
    </ul>
    

    This assumes that the view model for the main view has a collection of Menu objects in an observableArray called menus.

    It renders a <li> element for each Menu giving it a dropdown class.  The <a> element will trigger Bootstrap to open the <ul> element that immediately follows the <a>.  That <ul> is generated by a compose binding calling onto the Menu view to render that one Menu and all of its children recursively.

    Categories: JavaScript

    Tags: Durandal, Knockoutjs, KoLite

  • Building a Dynamic Application Menu with Durandal.js, Knockout, and Bootstrap (Pt. 1)

    I’m going to do a longer series here about how to create a dynamic menu bar system with Durandal, Knockout, and Twitter Bootstrap.  This menu bar is going to emulate the traditional desktop application menu bar you find in apps (like the File, Edit, or View menus, for example).  The special thing here is that it will be 100% dynamic.  This will allow interesting scenarios such as dynamically altering the application menu when the application is in a different mode or allow something like plug-ins to alter the menu structure adding new commands.

    We will use the following libraries/frameworks to perform this task:

    • We’ll use Bootstrap to style the menus and get basic drop down menu support.  The menus in Bootstrap look very pretty and work very well using just one .js file for behavior and a little bit of markup.
    • We’ll use Durandal to structure the application and to take advantage of the composition engine it has.  I assume you know how to get a basic Durandal project up and running so I’m not going to spend a lot of time discussing how Durandal works.
    • We’ll use Knockout to do all of the data binding.  Our menu items will have observable properties in them so that menus will dynamically change when you change things about them in code.
    • We’ll also make use of KoLite by John Papa which provides a simple KO extension (the command) to abstract the idea of a UI command.  A single menu item will wrap a ko.command().  If a command does not allow execution, the corresponding menu item(s) will not allow it and will appear disabled.  Also, when clicking on a menu item, the command will execute.  This will all happen through data binding and will not

    For this first part, let’s build the JavaScript object model for the menu system.  A Menu object (defined in ui/menu.js in my project) represents one menu on a menu bar (such as a File menu).  It will contain zero or more MenuItems which are the individual menu selections in the menu.  There is also a special menu item that acts as a divider or separator.  These dividers draw thin lines between menu items to help visually group commands.  They are not clickable.

    Here’s the code for the MenuItem class (defined in ui/menuItem.js as a Durandal module):

    define(function (require) {
        var MenuItem = function (itemText, command, items) {
            this.text = ko.observable(itemText);
            this.command = command || ko.command({ execute: function () { } });
            this.items = ko.observableArray(items || []);
            this.hasSubMenu = ko.computed(function () {
                return this.items().length > 0;
            }, this);
        };
    
        MenuItem.prototype.addMenuItem = function (menuItem, position) {
            if (position) {
                this.items.splice(position, 0, menuItem);
            }
            else {
                this.items.push(menuItem);
            }
        };
    
        MenuItem.prototype.addDivider = function (position) {
            var item = { divider: true };
            if (position) {
                this.items.splice(position, 0, item);
            }
            else {
                this.items.push(item);
            }
        };
    
        return MenuItem;
    });
    

     

    A menu item takes a menu item text (the text to appear in the menu), an optional KoLite command, and an optional set of child sub-items.  The sub-items are used for when the menu item is actually a menu within a menu and will be rendered with a an arrow to the right of the menu item using Bootstrap.

    A Menu class also exists as a top-level container for MenuItems.  Think of this as the File menu or Edit menu.  It is defined in ui/menu.js as a Durandal module:

    define(function (require) {
        var MenuItem = require('ui/menuItem');
    
        var Menu = function (text, items) {
            this.text = ko.observable(text);
            this.items = ko.observableArray(items || []);
        };
    
        Menu.prototype.addMenuItem = function (menuItem, position) {
            if (position) {
                this.items.splice(position, 0, menuItem);
            }
            else {
                this.items.push(menuItem);
            }
    
            return menuItem;
        };
    
        return Menu;
    });
    

     

    This class takes the text of the menu item (“File”) and the collection of MenuItems to add to the menu.  You can call addMenuItem() to add a menu item after the initial creation of the menu.  The position parameter will add the menu in the specified position.  If you don’t specify a position, it will be added to the end of the menu.

    In the next part of the series, we’ll look at the HTML and KO data-bindings that will render the menus.

    Categories: JavaScript

    Tags: Bootstrap, Durandal, Knockoutjs, KoLite

  • Tree Traversals with JavaScript

    Let’s do this one more time – this time with JS.  Here is a constructor function for a generic tree node that has one parent and possibly N children:

    var TreeNode = function (data) {
        this.parent = data.parent || null;
        this.children = data.children || [];
    };
    
    TreeNode.prototype.isLeaf = function () {
        return this.children.length == 0;
    };
    
    TreeNode.prototype.isRoot = function () {
        return this.parent == null;
    };
    

     

    Here is how we’d traverse the tree with a depth-first search (note that I’m using the underscore.js library for the each() function – you could also use ES5’s Array.forEach() if you do not want to support older browsers – or use a polyfill):

    function visitDfs(node, func) {
        if (func) {
            func(node);
        }
    
        _.each(node.children, function (child) {
            visitDfs(child, func);
        });
    }

    Doing a breadth-first traversal is also straightforward thanks to JavaScript’s Array class and its queue/stack-like functionality:

    function visitBfs(node, func) {
        var q = [node];
        while (q.length > 0) {
            node = q.shift();
            if (func) {
                func(node);
            }
    
            _.each(node.children, function (child) {
                q.push(child);
            });
        }
    }
    

     

    Finally, let’s do something tangible with a depth-first traversal.  In the next code snippet, the prune() function will prune out nodes that are single children to their parents in order to collapse the tree into a simpler structure.  This will eliminate groupings of nodes that aren’t really grouping anything because their branching factor is 1:

    function prune(root) {
        visitDfs(root, function (node) {
            if (node.isRoot()) {
                return; // do not process roots
            }
            if (node.children.length == 1 && !node.children[0].isLeaf()) {
                var child = node.children[0],
                    index = _.indexOf(node.parent.children, node);
                node.parent.children[index] = child;
                child.parent = node.parent;
            }
        });
    }
    

     

    This code works by doing a DFS and calling a function on each node.  That function examines the node to see if it has exactly one child and that the one child node is not a leaf (we don’t want to prune out leaves or roots).  If it is, the child node is extracted out and assigned to the node’s parent’s children and then the child’s parent is reassigned to the node’s parent.  This effectively eliminates (prunes) the node out of the tree collapsing it into a simpler structure.

    Categories: JavaScript

  • Another Technique for Tree Traversals

    In the previous post, I talked about doing a DFS on a rooted tree using the C# foreach loop/iterator concept.  In this post, I will discuss a different technique that is more function-based.

    A common task on a tree is to traverse it in some defined order – pre-order, level-order, post-order, etc.  When we visit a node, we’d like to perform some action on it.  We can use delegates in C# (reference to a function) and lambdas to make this really compact and elegant:

    public static void VisitDfs(TreeNode node, Action<TreeNode> func)
    {
        if (func != null)
            func(node);
        foreach (TreeNode child in Children)
        {
            VisitDfs(child, func);
        }
    }
    

     

    If you pass in the root node to this method, you can call the function you send in on each node in that order.  You can apply this concept for other traversal patterns too:

    public static void VisitBfs(TreeNode node, Action<TreeNode> func)
    {
        Queue<TreeNode> q = new Queue<TreeNode>();
        q.Enqueue(node);
        while (q.Count > 0) 
        {
            node = q.Dequeue();
            if (func != null)
                func(node);
            foreach (TreeNode child in Children)
            {
                q.Enqueue(child);
            }
        }
    }
    

     

    This performs a breadth-first traversal (also called a ‘level order’ traversal with trees), calling the specified function at each node visit.

    Here’s a sample usage of the DFS to count the number of leaves in a tree:

    public static int CountLeaves(TreeNode root)
    {
        int leafCount = 0;
        VisitDfs(root, node => {
            if (node.Children.Count == 0) {
                leafCount++;
            }
        });
    
        return leafCount;
    }
    

    Categories: C#

  • Performing a DFS over a Rooted Tree with a C# foreach Loop

    I’ve been working with trees a lot lately and one thing that occurs quite frequently in my problem space is the action of performing a depth-first search (DFS) or a 'pre-order traversal’.  In a DFS, you visit the root first and then search deeper into the tree visiting each node and then the node’s children.
     

    Since this is so common, I’d like a routine that helps me do that very simply.  What I’d really like to do is simply do a foreach loop over the tree and with each iteration pull out the next node that would satisfy a DFS.  This is where the C# yield keyword comes into play.

    The following snippet presents a TreeNode class which is just a generic node in a tree that has N children:

    public class TreeNode
    {
        public TreeNode()
        {
            Children = new List<TreeNode>();
        }
    
        public TreeNode Parent { get; set; }
        public List<TreeNode> Children { get; private set; }
        public bool IsLeaf { get { return Children.Count == 0; } }
        public bool IsRoot { get { return Parent == null; } }
    
        public IEnumerator<TreeNode> GetEnumerator()
        {
            yield return this;
            foreach (TreeNode child in Children)
            {
                IEnumerator<TreeNode> childEnumerator = child.GetEnumerator();
                while (childEnumerator.MoveNext())
                    yield return childEnumerator.Current;
            }
        }
    }
    

     

    Most of the functionality is in the GetEnumerator() method.  When a class in C# has a method called GetEnumerator() that returns a type IEnumerator or IEnumerator<T>, you’ll be able to use that type in a foreach loop.  Also notice that the type returned by my method does not return IEnumerator<TreeNode> but rather just TreeNode.  The C# compiler will fill in the proper details automatically.

    In order to do a DFS, we first visit the node itself by returning it (this) in a yield statement.  When the iterator executes the next loop, the GetEnumerator() method remembers the execution state of the method and picks up right where it left off.  At that point, we enter into a loop over the Children collection.  We basically repeat the enumerator process over this collection manually by doing a while loop where we exit the loop if no more children exist.  In the body of this while loop, we execute another return within a yield, returning each child node.

    To use this enumerator to perform a DFS, you’d foreach over the root node of your tree (I’m assuming a singly rooted-tree here):

    foreach (TreeNode node in root)
    {
      // perform action on node in DFS order here
    }
    

    Categories: C#

  • Using ASP.NET MVC and Shibboleth Authentication

    If you’re using Shibboleth as your resource authentication method, there’s a few things you need to do in order to use it effectively with ASP.NET MVC.

    The first thing to do is to configure routing correctly.  Shibboleth uses a special route against your website that is intercepted by the ISAPI filter that is part of the Shibboleth install on your web server.  However, in integrated pipeline mode, your application will receive these Shibboleth requests and won’t know what to do with them and issue a 404 not found.  The solution is to ignore the routes that Shibboleth uses and let them pass through to the Shibboleth system:

    routes.IgnoreRoute("Shibboleth.sso/{*pathInfo}");
    

     

    The ‘routes’ variable is a RouteCollection type passed to your RegisterRoutes() method in the RouteConfig.cs file of an MVC 4 project.  This ensures that the Shibboleth routes are never processed by your application.

    The second thing is to turn off all authentication for your web site.  Since Shibboleth’s single sign-on is going to handle protecting your resources, you will not need to configure any authentication in your web.config or in IIS.  Instead, you will configure it with your shibboleth2.xml file.

    Categories: ASP.NET MVC

  • RavenDB Survival Tip #1: Indexing Gotcha

    This series of posts is going to cover some things I’ve learned using RavenDB, a document database solution for .NET. 

    I was working on a class (a User class) where I needed to add a property to the class that didn’t exist in the original version of the application.   So, I went and added it to the C# class (User) and I also updated an Index in RavenDB to use this new property so that I could filter by it in a query.  It built and when I deployed I could see that Raven was updating the index and did so in a timely manner.

    I then went and wrote some code to filter by that new property in a query.  The property I had added was a boolean type and so all of the models that I had in the database (some 50,000+ User documents) would just get the default value of false for this new property.   However, when I went to run this query, no documents were returned.  What happened was that when you add a new property to a model, it isn’t going to go and update all of your existing documents with a property value even if a default is implied or provided (as is the case with a bool being false by default).  There is simply no property in the JSON store in the User document that matches this property I was querying by. 

    The solution is that when you do something like that, you need to run a Patch in RavenDB.  This will ensure that all of your documents get synced to the new additional property you have made.  Perhaps if this was some integer or string property that had to be set to something specific I would have picked up on the problem before even attempting the query. 

    Categories: RavenDB

  • First Impressions on Durandal

    I’ve been working with Durandaljs for a few weeks now.  It’s an open-source single page application (SPA) framework written to use technologies that you might already know.  So far, I’m really impressed and I’ve enjoyed working with it.  Since I already do a fair bit of Knockout development, I like the fact that Durandal uses KO and several other things I know such as Requirejs.

    I’ve made SPA-like parts to web apps before but I always found it hard to manage all of the bits and pieces that go together to make an app.  For example, modal dialogs were typically a pain to implement.  In Durandal, it basically handles the management of the modal’s appearance and how its view is retrieved and dynamically bound to the view model that you create.  The development of a Durandal app basically goes like this:

    • Break up your app into views with complementary view models (the views are HTML files with KO bindings and the view models are just JavaScript classes or object literals).  The view model is written as one RequireJS module.  The modularity of your application is one of the great strengths of this framework and really is more of a RequireJS thing than a Durandal thing.  But Durandal does tie all of it together so think of it as a coordinator between libraries rather than its own implementation. 
    • You then setup your navigation and routing.  Currently, Durandal is route-engine agnostic and supports a plug-in model to support routing.  It ships with a SammyJS-based routing engine which is pretty easy to use.  This is one area that I personally lacked some knowledge in, but SammyJS is pretty easy to learn to do most common tasks.
    • Once you get views and view models together, you can move on to advanced stuff like view composition and eventing in Durandal.

    I think one of the biggest strengths of Durandal is that it has a very powerful view composition system.  This basically allows you to find views, merge them with a view model and inject them into your DOM with ease.  You can use composition to swap out or render sub-sections of a page (such as a tab content area) or you can even do awesome things like recursive composition to build tree views or menus with sub-menu capabilities.

    If you are a Pluralsight member, I suggest checking out John Papa’s course on SPAs where he talks about Durandal (and many other aspects of SPAs). 

    The only bad part so far for me is that I wish I had known about this framework six months ago to use it in my current project.  I am currently thinking of refactoring to using Durandal.

    Categories: JavaScript

  • JavaScript Libraries and Frameworks

    I’ve been doing a lot of client-side development lately.  Our web browsers are becoming more capable with each new version.  Browser inconsistencies are disappearing behind standards, polyfills, and abstractions in good JavaScript libraries.  I thought that in this post I would list all of the JavaScript libraries that I use or have used and found to be quite good.  These libraries are extensible, documented, and are well-tested.  They have worked for me in professional and personal projects.  I will probably be adding to this list over time:

    • Durandal.js – a SPA (single page application framework) that adds powerful view composition.  It uses what you might already know (like Knockout.js, SammyJS, and RequireJS).  If you are building a SPA and aren’t using a formal framework like AngularJS, you should check this out.
    • Knockout.js – an MVVM (model-view-viewmodel) library for the web.  This adds data-binding capabilities to your app so your UI stays in sync with your JavaScript models through two-way data binding.  Also known as KO.
    • KoLite – this is small collection of KO extensions that I find useful.  In particular, I like the “command” binding handling that adds a way to add XAML like ICommand-ness to KO. 
    • RequireJS – I use this in conjunction with Durandal as it is a requirement of Durandal.  It allows you to organize your code into “modules” – a programming feature lacking from JS.
    • SammyJS – this is a client-side routing engine that is a default routing plugin for Durandal. 
    • jQuery and jQuery UI – everybody knows these two.
    • toastr – a small and effective notification library that “toasts” a message to the user.  It is so simple and looks very beautiful too.
    • accounting.js – this small library helps with parsing and formatting monetary amounts in JS.  I use this a lot with professional projects because most of the work I do involves capturing and displaying monetary amounts (as do a great deal of applications).
    • mustache.js – a small string template system for when you want to generate HTML using string templates (as opposed to Knockout’s or Angular’s DOM-based templating).  I use this only when I need to work with a third-party control that allows custom templating but templates must be delivered as strings.  In general, DOM templates tend to be better in my opinion.
    • date.format.js – a small date/time parsing and formatting library that works quite well.  I also hear that moment.js is really good, but have not used it.  JS’s support for Dates is pretty bad so these libraries help ease the pain, considerably.

    Most of these libraries have packages on NuGet.  Some are on npm too.

    Categories: JavaScript