Comments on lua unit test frameworks

April 9th, 2010

I’ve been writing my unit tests with assert(), and it works ok.

But, I’d like to split my test suite across multiple files, I’d like to run all or just a part of the suite, and I’d like, when a test fails, to rerun just that specific test, or tests matching a pattern.

So, I need a better tool, and I want to choose one I can stick with for a while. I looked around at what I could find, starting with the ones listed on the lua wiki.

Analysis is below, but the conclusion is that lunit was the winner.

Still, one feature wasn’t working on the master, and I had some trouble finding a good way to integrate into some of our codebases. So, I forked it, and pushed some changes to github.

lunit.lua
---------

- Source: 

- Has a weird shell-script wrapper, but not necessary to use,
just call lunit.main().

- The lunit script fails for me (down.lua) with bizarre error
 messages when using -t, not sure why, and its construction
 makes it hard to figure out!

- Distinguishes between error/assert and it's internal assertions.

- Doesn't accept a fmt string  + .. for the assert msg

- Should support a -t argument, but I'm having trouble using
it from lunit. It does work from lunit.main(), but irritatingly
 involves matching the entire test name, which I fixed.

lunity.lua
----------

- Source: 

- Seems to be a not-well-justified clone of lunit, basically
feature  compatible, with slightly different names, and two
missing features.

- There isn't any way to report over tests from multiple
modules, other than just running them individually.

- Doesn't distinguish between errors (calling nil, bad calls,
etc.) and unit test assertions.

moonunit
--------

- Source: 

- Too simple to be considered.

luaunit
-------

- Seems like a port of pyunit's OO framework, unnecessarily heavyweight for lua.

shake
-----

Too weird, and doesn't allow running single testcases.

...
---

All the behaviour and spec driven were way too heavy-weight,
 and syntactically noisy. Also, a decent BDD or spec-based
 framework should layer on _top_ of a lower-layer of unit
tests, so it can be combined with other approaches.

Deploying Sinatra apps with CGI

March 1st, 2009

First of all, either hack out the puts statements, as described for FCGI here or wait for the new release that should fix this.

Anyhow, you can do all that crazy vlad or capistrano thingy, or do this.

Make an .htaccess:

# My gems are in ~/.gem, but rubygems won't find them if HOME
# isn't defined, and on a shared host, Apache's HOME is not mine.
SetEnv HOME /home/octet
 
RewriteEngine On
RewriteBase /
RewriteRule ^index.cgi - [L]
RewriteRule ^(.*)$ index.cgi/$1

Make an index.cgi:

#!/usr/bin/env ruby
 
require 'rubygems'
 
# So $0 is the "app_file" and we get run.
require 'sinatra'
 
$:.unshift File.dirname(__FILE__) + "/lib"
 
# So when Sinatra auto-loads, our routes get reloaded.
load 'lib/vpim/agent/app.rb'

Leave the above stuff there. You don’t have to change it, you don’t have to deploy it. Its working.

My app is going into lib/vpim/agent/app.rb. So, in my local Makefile at the route of my project (yeah, the cool kids use Rake, but they don’t get two line deployment, either), add this:

deploy:
  rsync -v --archive --compress --cvs-exclude --exclude=.svn/ --del lib octet:webapps/agent/

In vim, “:make deploy” will deploy your app.

Enjoy.

Sam

Sinatra – the undocumented bits

March 1st, 2009

So, Sinatra has lots of docs, but most of it is repeated examples of how to do simple things.

You basically have to read the code to figure out how things work. Here’s some of my notes.

Undocumented Sinatra
====================

Which call?
-----------

Sinatra has two implementations of #call, the interface that rack uses to
pass HTTP requests, and they behave very differently!

- Sinatra::Base#call (Sinatra::Application inherits from Base)

#dup's itself, sets @env to rack's env, builds a rack Request and Response
(@request, @response), and calls dispatch!, which calls route!.

In route!, @params is set, self.class.filters are evaluated (... what are filters?).

Then, routes are iterated, and the first where the pattern and conditons match has
it's block instance_eval'ed.

- Sinatra::Base.call

If Sinatra is running itself, its behaviour can be quite different.

It starts by calling Base.run!, this will detect the rack handler to use (base
on existence, it doesn't do anything clever like rackup does), and run it with
the Application *class* as the app. That means the Base.call class-method will
be used, not the Application instance's one...

This "call" is the one that (optionally) will reload the app_file, it is also
the one that optionally uses various rack middleware (Session::Cookie,
CommonLogger, MethodOverride), as well as any that were set (set :middleware,
...). Only after this, does it call #call.

If you don't arrange for this "call" to be the one Rack uses, you won't see any
of those behaviours. I.e., reloading will never happen.

In summary, these lines do VERY different things when using Sinatra with Rack:

  run Sinatra::Application

and

  run Sinatra::Application.new

Development Mode
----------------

The "environment" can be set :deployment, :development, or :test. It
defaults to :development when RACK_ENV isn't set.

  => What is supposed to set RACK_ENV? It looks like something Rack might set,
  but it is not.

app_file is determined how?

  when sinatra/main is required, it looks through the call stack to find
  the name of file doing the require "sinatra", and uses it (if it thinks
  its not a library function), or uses $0

run? => true if $0 == app_file

The app_file is the one that is auto-loaded...

Sinatra::Application gets run! at exit, if run?

If run?, ARGV is auto parsed: args are -e, -s, -p, and -x.

If run?, then handler is detect_rack_handler, which just uses the first one
that is found... Thin, Mongrel, Webrick.

 ==> Would it be better to do as rackup does, and choose based on environment,
 if possible? Then I could take my single-file app:

-----
#!/usr/bin/env ruby

get "/" { "hi" }
-----

Auto-Loading
------------

Sinatra and Rack implement auto-reloading during development mode in very
different ways. To make them work, you need to do almost the exact opposite.

With Sinatra, the global state gets reset, and the app_file gets loaded
(Kernel#load). This is why if you have any code used by your app_file that
isn't actually inside it, you need to load it, NOT require it. You also need to
arrange to require "sinatra" in the file that you want to be the app_file.
Rackup users note this well!

With Rack and ``use Rack::Reloader, 0'' , it works by going through all files
that have been required, and seeing if the on-disk file has been updated by
checking the file time. So, you MUST require anything you depend on that has
changed.

You might want to consider using both mechanisms. Sinatra will rebuild its
routing table with every request, and if you use the Rack reloader all your
class definitions in required files will get reloaded.

Rackup – the undocumented bits

January 25th, 2009

I struggled a bit with Rack. Once I gave in, stopped looking for docs, and just read the code, it made more sense. Since I’ve a bad memory, I wrote down what I found.

This is now up on the rack wiki, hope it helps someone.


Rackup Howto
============

Config file syntax
------------------

The config file is config.ru if none is specified.

Handling of config files depends on whether it's .ru, or something else. Its
important to define the application that rackup will run correctly, failure to
do so will result in mysterious runtime errors!

* .ru:

The config file is treated as if it is the body of
	app = Rack::Builder.new { ... config ... }.to_app

Also, the first line starting with '#\' is treated as if it was options,
allowing rackup arguments to be specified in the config file. For example:

	#\ -w -p 8765
	use Rack::Reloader, 0
	use Rack::ContentLength

	app = proc do |env|
	  [ 200, {'Content-Type' => 'text/plain'}, "a" ]
	end

	run app

	# vim:ft=ruby

Would run with Ruby warnings enabled, and request port 8765 (which will
be ignored unless the server supports the :P ort option).

* .rb, ...:

The config file is required. It must assign the app to a global constant so
rackup can find it.

The name of the constant should be config file's base name, stripped of a
trailing ".rb" (if present), and captitalized. The following config files all
look for Config: ~/bin/config, config.rb, /usr/bin/config, example/config.rb.

This will work if the file name is octet.rb:

	Octet = Rack::Builder.new do
	  use Rack::Reloader, 0
	  use Rack::ContentLength
	  app = proc do |env|
	    [ 200, {'Content-Type' => 'text/plain'}, "b" ]
	  end
	  run app
	end.to_app

	# vim:ft=ruby

Auto-selection of a server
--------------------------

The specified server (from Handler::get) is used, or the first of these
to match is selected:

* "PHP_FCGI_CHILDREN" is in the process environment, use FastCGI
* "REQUEST_METHOD" is in the process environment, use CGI
* Mongrel is installed, use it
* Otherwise, use Webrick

Automatic middleware
--------------------

Rackup will automatically use some middleware, depending on the environment you
select, the -E switch, with "development" being the default:

- development: CommonLogger, ShowExceptions, Lint
- deployment: CommonLogger
- none: (none :-) 

CommonLogger isn't used with the CGI server, because it writes to stderr, which
doesn't interact so well with CGI.