Ruby AND precedence

And you thought and was equal to &&

Have you ever looked at ruby code and said wow, this is so readable, I
can use

and

instead of &&, how cool! Well, not
quite. There are differences and pretty important ones to consider.
Here, I’ll show you.

The most basic use of and:

	a = "b" and "c"
	puts a # => "b"
	a = "b" && "c"
	p a # => "c"

Notice the precedence issue? With

and

, the
precedence is on the left-hand side while

&&

runs through the entire phrase.

Why is this important? And and && phrases, in practice are generally
used in if statements and this would cause a big issue with

and

statements. Why? (Example time)

	1 == 3 and true # false
	1 == 3 && true  # false
	true and 5 == 2 # true
	true && 5 == 2  # false

Did you catch that?

	true and 5 == 2 # true

It’s clearly not the logic you’d expect when you are using and… One
more just to prove my point

	"hello" && e = "world" 	# world + world
	"hello" and e = "world" # hello + world

AWS elastic Mapreduce

Amazon Webservices (AWS) have just announced their Elastic Mapreduce webservice. With their massive architecture, cloud-monopoly (not quite) and the apparent ease of use of the service, the only way this can possibly fail is that the average consumer probably has no idea what mapreduce is… You can catch me playing it with later this week (after http://larubyconf.com/)

Happy Squared Day!

Today is 3^2 = 9 day!

Math nerds celebrate

Easy configuration with Erlang

Erlang isn’t the easiest of languages to pass around a set of configuration. You can always write a gen_server to handle holding configuration for you, but why? That’s just another running process and what happens if it dies? You have to account for that, etc. 

I’ve been working on a framework stack in Erlang for a current project of mine and I’m delighted to share it for posterity.

Introducing: Configerl.

Configerl is my library I use for projects to pull out configuration. I’ll show you what I mean:

Say I have a list of tuples intended for configuration purposes, like so:

Config = [{port, 12345}, {name, "converse"}].

With configerl you can pull out and parse the different configurations easily:

config:parse(port, Config) => 12345.

What about getting with a default?

config:parse_or_default(port, Config, "ted").

Yes yes, nothing so grand about that, but what about if you want to merge configurations?

config:append([{port, 12345}, {name, "converse"}], [{friends, "whisper"}]).

What about updating configurations:

config:update({port, 54321}, Config).

How about removing a configuration:

config:delete(port, Config).

And now we get to the really fun stuff:

Fetching multiple keys

[Port, Name] = config:fetch([port, name], Config).

Fetching multiple keys with defaults

[Port, Name] = config:fetch_with_default([port, name], Config, [12345, "converse"]).

Fetching multiple keys with a default configuration

DefaultConfig = [{port, 12345}, {name, "whisper"}].
[Port, Name] = config:fetch_with_default_config([port, name], Config, DefaultConfig).

The project is still growing quite a bit and is fully tested up to this point. Please feel free to add to the library or do with what you please.

Awesome!

Hello AT&T interactive

Yes, I know it has been quite a while since I’ve written a post and for good reason. I’ve been quite busy.

Starting at the end of this month, I will be taking a new step in my programming career as a Lead Software Engineer at AT&T Interactive’s R&D department, continuing development on PoolParty.

CitrusByte has been a great company to work with, the developers are super smart, the culture is fun and the work is dynamic. It was not an easy decision to make as I have thoroughly enjoyed my experience and I will miss working with the crew.

I look forward to continuing my work on PoolParty with AT&T Interactive and will soon resume my blogging at regular intervals once again.

Introducing Dslify!

One of the earliest piece of functionality I started working on when I started working on PoolParty was the DSL syntax. Because I wanted the syntax to be light enough where simple inheritance would allow you to introduce new class types and parented models, I wanted to be able to give any method on any object and be able to retrieve it later. The syntax, which looks something like:

cloud :name do
 instances 2..5
end

is so terse and readable, I’ve extracted the functionality into a gem!

Now, you can add your own DSL syntax to any class by adding 1 line to your class! And, you get default syntaxes for free!

Let’s add one for a fake class that we can use to look up showtimes!

First, the class used to look like this:

class MovieFone
end

To add the DSL syntax, we add the 1 line:

class MovieFone
 include Dslify
end

Now, any method that is called on the class MovieFone that doesn’t exist, will be called on the instance. These methods and their attributes are all stored in a method called: __options (double underscores to avoid any collisions with user defined option methods). Now you can fetch any method called on your instance (with or without using =) on the options, or simply by calling the method.

Example? Of course!

class MovieFone
 include Dslify
end

mf = MovieFone.new
mf.movie "Can't buy me love"
mf.times 8.pm..10.pm

# Or, my preferred syntax:

class MovieFone
 def initialize(&block)
  instance_eval block if block
 end
end

MovieFone.new do
 movie "Lucas"
 times 8.pm .. 10.pm
end

One neat feature that may not be apparent is that you can support different runtime methods with the same call, but only have to evaluate it at the runtime. For instance, say you want to give either a range of times (like above) or accept an array of times you are available:

mf.times = 8.pm, 10.pm, 12.am

Simple! When you setup the search method, the argument will be of the class type you send it! Cake!

Find the gem here: http://github.com/auser/dslify/tree/master!

Using Capistrano with EC2

It has long been known that capistrano is borked when using EC2 because it requires the public key.

The work-around has been to download the public key from EC2 before running capistrano, a very ugly solution indeed. Well, after digging into the net-ssh source, it’s obvious that the public key was only used to hold identities in a hash. Because net/ssh uses ruby’s openssl library, we can call the public key to be generated from the private key, rather than requiring it from the user!

So, without further ado, no longer do you need to snatch up the public key from your EC2 instance. The fork is located at: http://github.com/auser/net-ssh/tree/master

Git-style binaries

In this version of PoolParty internals, we’ll check out the git-style binaries used in PoolParty. Note, for this episode, I may move a little fast as there is a lot to cover. 

Git, if you don’t already know is a super sleek SCM (Source control management system). It gives you hundreds of binaries using the idiom: git [action] options command… One big benefit to this is that all the binaries are self-contained, but function together. 

With PoolParty, it makes sense to contain the binaries, both to enable quick development and to target the action the command-line method will be doing. This enables us to do cloud [action], such as cloud start and cloud ssh. How do we do this? Let’s dive in!

Opening poolparty/bin/cloud, we see the following

#!/usr/bin/env ruby
require "poolparty"
require "poolpartycl"

name = ARGV.select {|arg| arg
    if Binary.available_binaries_for("cloud").include?(arg) }.first

# Hiding methods here that strip out actions that aren't included
# in the PoolParty list of available command-line actions
# for brevity

# If no command is passed in, show help
new_args.push("-h") unless name

o = PoolParty::Optioner.new(new_args,
  {:extra_help => "Cloud actions\n#{Binary.list_binaries_for("cloud")}",
  :abstract => true}) do |opts, optioner|
  opts.on('-n cloudname', '--name name', 'Address this cloud')
    { |c| optioner.cloudname c }
end

program_name = "#{File.basename($0)}-#{name}"
program_location = File.join(Binary.binary_directory, program_name)

command_line = "#{program_location}"

# Run it from the command-line first
if Binary.available_binaries_for("cloud").include?(name)
  system command_line, *ARGV
else
  puts "Unknown poolparty binary: #{name}"
end

Phew, that’s a lot. It’s super simple, the first few lines are ensuring that the action is contained in the list of available actions.

These are gathered just by looking in the directory with the binaries that start with “cloud,” in this case:

Dir["#{binary_directory}/#{ty}-*"].map
 {|a| File.basename(a.gsub(/#{ty}-/, '')) }.sort

Moving on, the Optioner is a custom optsparser class that sits inside of PoolParty that gives PoolParty the ability to extend it’s own option parser. Not much to the class really, but I urge you to check out the class if you are interested on knowing how it works.

Finally, we are going to call the command! We have to rebuild the arguments sent in such that the arguments are parsed properly by the next binary that will consume them. We can use the splat operator so that when the optsparser reads them again, it’s as though the arguments were entered on the command-line.

Phew, that was quite a bit! As always, I urge you to check out PoolParty if you haven’t already. It’s pretty spiffy and super fun!

Talk to your cloud directly with ruberl

Want to communicate with your cloud quickly?

Easy! First, download the gem ruberl

gem install auser-ruberl

Now you just have to set host and port and you can communicate with your gen_tcp server (or, in PoolParty’s case, the messenger) with either messenger_send! or messenger_cast! like so:

require "ruberl"

class Test < Ruberl::Base
end

@t = Test.new("75.101.162.232", 7050)
loop do
  nodes = @t.messenger_send!("get_current_nodes")
  load = @t.messenger_send!("get_current_load cpu").to_f
  puts "Load on nodes #{nodes}: #{load}"
  sleep(10)
end

Alternatively, you can just set the attr_accessors host and port within your class and do the same thing, for instance

class Test < Ruberl::Base
 ... # some more methods
 def get_load
  @host = "host"
  @port = "port"
  messenger_send!("get_current_load cpu")
 end
 ...
end

Stay tuned for more quick tips, internals and discussions!

Inherited resourcing and the power of subclassing

In this edition of PoolParty internals, I’m going to show you how the basic resourcing is done. Virtual resourcing is out of the scope of this blog post, but we’ll come back to that on another blog post.

If you are not familiar with PoolParty, I urge you to check it out

From within PoolParty, resources are a cornerstone. Most all the resources implemented in PoolParty have the same basic structure of each other, but require different parameters. For instance, a file resource can have content associated with it, while an exec cannot. However, they both behave like each other in the context of usage.

Most of the resource classes in PoolParty are tiny. For instance, the Host resource is 4 lines long without a method defined.

module PoolParty
  module Resources        

    class Host < Resource
      default_options({
        :name => "$hostname",
        :ip => "$ipaddress"
      })
    end

  end
end

When the following code is in the pool.spec, the a new host resource is created (we’ll see later that it means a new instance of the Host class is instantiated) and stored in the containing class.

	has_host(:name => "orca", :ip => "67.125.84.125")
	# Note that this is the same as calling
	host(:name => "orca", :ip => "67.125.84.125")

Almost all of the resources implementation are that small. So how do we do this?

	def self.inherited(subclass)
	end

Because we already use the method_missing to create methods from on the cloud to keep our DSL clean, we will use inherited to create the methods we want on our containing class as well as to build a list of the available resources. Let’s look at the method currently implemented in PoolParty:

def self.inherited(subclass)
  subclass = subclass.to_s.split("::")[-1] if subclass.to_s.index("::")
  lowercase_class_name = subclass.to_s.underscore

  # Add add resource method to the Resources module
  unless PoolParty::Resources.respond_to?(lowercase_class_name.to_sym)
    method =<<-EOE
      def #{lowercase_class_name}(opts={}, parent=self, &blk)
        add_resource(:#{lowercase_class_name}, opts, parent, &blk)
      end
      def get_#{lowercase_class_name}(name)
        get_resource(:#{lowercase_class_name}, name) if in_a_resource_store?(:#{lowercase_class_name}, name)
      end
    EOE
    PoolParty::Resources.module_eval method
    PoolParty::Resources.add_has_and_does_not_have_methods_for(lowercase_class_name.to_sym)

    available_resources << subclass
  end
end
def self.add_has_and_does_not_have_methods_for(type=:file)
  module_eval <<-EOE
    def has_#{type}(opts={}, parent=self, &block)
      #{type}(#{type == :exec ? "opts" : "{:is_present => ''}.merge(opts)"}, parent, &block)
    end
    def does_not_have_#{type}(opts={}, parent=self, &block)
      #{type}(#{type == :exec ? "opts" : "{:is_absent => ''}.merge(opts)"}, parent, &block)
    end
  EOE
end

I’m going to make this quick, but feel free to stop me and ask me if something is confusing…

Whenever a class is subclassed, the parent class receives a callback hook called “inherited.” This method is called on any class that gets subclassed, for instance:

class A
  def self.inherited(sublcass)
    puts "inherited from #{sublcass}"
  end
end
class B < A
end

# inherited from B

This way, in order to add a resource to PoolParty, all we have to do is subclass the PoolParty::Resources::Resource class.

I’m making sure that we only have to run this method once, so that both the method doesn’t get uselessly overwritten and second that it’s nicer on GC. Second, we are adding two methods at the time to the resources (note, that clouds, pools, and plugins all include the Resources module, thus all these methods are allowed within those contexts.

Finally, we add the methods onto the class, add the add_has_and_does_not_have_methods_for methods to the class (this is where the has_ and does_not_have methods get added).

One thing you’ll note from this is that every single resource created can be called 3 ways, resource_name, has_resource_name and does_not_have_resource_name. Finally, we add it to the available_resources array just so we keep track of these.

I hope that works for you and that you can find a use for self.inherited.

Update: Stay tuned for a special inside PoolParty about rspec extensions… pretty hot stuff, IMHO.