Ari Lerner

Wounded

2010-02-02T00:00:00-08:00

You used to speak so easy,
Now you're afaid to talk [to me].
Its like walking with the wounded.

Carrying that weight way too far, 
The concrete pulled you down so hard
Out there with the wounded,
We're missing you.

I know.

A thought...

2009-10-20T00:00:00-07:00

ruby -le'32.times{|y|print" "*(31-y),(0..y).map{|x|~y&x>0?" .":" A"}}'

Try it!

New Blog

2009-07-14T00:00:00-07:00

Check out my new code blog: http://xnot.org and the latest blog post about erlang and Makefiles to simply erlang development!

Accomplice, Hollywood

2009-06-27T00:00:00-07:00

My baby brother is in town for holiday for the summer and is visiting me in Hollywood this week. Sam and I took Jeremy to Accomplice, the improvised mystery on the streets of Hollywood. Originating in NYC, Accomplice takes show-goers around the Hollywood Blvd. to seemingly unrelated, uninvolved bystanders wrapping them up into the story.

Carefully choosing my words here as to not give anything away, anyone on the Boulevard can be involved and clues are scattered throughout the day. Even the fellow attendees in our group suspected us to be apart of the show, because we solved a few clues.

I must say that the experience was one I won’t soon forget and will most definitely recommend to friends. They do corporate events too! Highly recommend, even for birthday parties!

Ruby's precedence

2009-04-08T00:00:00-07:00

And you thought and was equal to &&

Have you ever looked at ruby code and said wow, this is so readable, I can use and instead of &&, how cool! Well, not quite. There are differences and pretty important ones to consider. Here, let’s see:

The most basic use of and:

  a = "b" and "c"
  puts a # => "b"
  a = "b" && "c"
  p a # => "c"

Notice the precedence issue? With and, the precedence is on the left-hand side while && runs through the entire phrase.

Why is this important? And and && phrases, in practice are generally used in if statements and this would cause a big issue with and statements. Why? (Example time)

  1 == 3 and true # false
  1 == 3 && true  # false
  true and 5 == 5 # true
  true && 5 == 2  # false

Did you catch that?

  true and 5 == 5 # true

It’s clearly not the logic you’d expect when you are using and… One more just to prove my point

  "hello" && e = "world"  # world + world
  "hello" and e = "world"   # hello + world

AWS releases MapReduce

2009-04-02T00:00:00-07:00

Amazon Webservices (AWS) have just announced their Elastic Mapreduce webservice. With their massive architecture, cloud-monopoly (not quite) and the apparent ease of use of the service, the only way this can possibly fail is that the average consumer probably has no idea what mapreduce is… You can catch me playing it with later this week (after http://larubyconf.com/)

Happy Squared Day

2009-03-03T00:00:00-08:00

Today is 3^2 = 9 day!

Math nerds celebrate

Easy configuration with erlang

2009-02-19T00:00:00-08:00

Erlang isn’t the easiest of languages to pass around a set of configuration. You can always write a gen_server to handle holding configuration for you, but why? That’s just another running process and what happens if it dies? You have to account for that, etc.

I’ve been working on a framework stack in Erlang for a current project of mine and I’m delighted to share it for posterity.

Introducing: Configerl.

Configerl is my library I use for projects to pull out configuration. I’ll show you what I mean:

Say I have a list of tuples intended for configuration purposes, like so:

Config = [{port, 12345}, {name, "converse"}].

With configerl you can pull out and parse the different configurations easily:

config:parse(port, Config) => 12345.

What about getting with a default?

config:parse_or_default(port, Config, "ted").

Yes yes, nothing so grand about that, but what about if you want to merge configurations?

config:append([{port, 12345}, {name, "converse"}], [{friends, "whisper"}]).

What about updating configurations:

config:update({port, 54321}, Config).

How about removing a configuration:

config:delete(port, Config).

And now we get to the really fun stuff:

Fetching multiple keys

[Port, Name] = config:fetch([port, name], Config).

Fetching multiple keys with defaults

[Port, Name] = config:fetch_with_default([port, name], Config, [12345, "converse"]).

Fetching multiple keys with a default configuration

DefaultConfig = [{port, 12345}, {name, "whisper"}].
[Port, Name] = config:fetch_with_default_config([port, name], Config, DefaultConfig).

The project is still growing quite a bit and is fully tested up to this point. Please feel free to add to the library or do with what you please.

Awesome!

Hello AT&T interactive

2009-01-14T00:00:00-08:00

Yes, I know it has been quite a while since I’ve written a post and for good reason. I’ve been quite busy.

Starting at the end of this month, I will be taking a new step in my programming career as a Lead Software Engineer at AT&T Interactive’s R&D department, continuing development on PoolParty.

CitrusByte has been a great company to work with, the developers are super smart, the culture is fun and the work is dynamic. It was not an easy decision to make as I have thoroughly enjoyed my experience and I will miss working with the crew.

I look forward to continuing my work on PoolParty with AT&T Interactive and will soon resume my blogging at regular intervals once again.

Introducing DSLify

2008-12-21T00:00:00-08:00

One of the earliest piece of functionality I started working on when I started working on PoolParty was the DSL syntax. Because I wanted the syntax to be light enough where simple inheritance would allow you to introduce new class types and parented models, I wanted to be able to give any method on any object and be able to retrieve it later. The syntax, which looks something like:

cloud :name do
 instances 2..5
end

is so terse and readable, I’ve extracted the functionality into a gem!

Now, you can add your own DSL syntax to any class by adding 1 line to your class! And, you get default syntaxes for free!

Let’s add one for a fake class that we can use to look up showtimes!

First, the class used to look like this:

class MovieFone
end

To add the DSL syntax, we add the 1 line:

class MovieFone
 include Dslify
end

Now, any method that is called on the class MovieFone that doesn’t exist, will be called on the instance. These methods and their attributes are all stored in a method called: __options (double underscores to avoid any collisions with user defined option methods). Now you can fetch any method called on your instance (with or without using =) on the options, or simply by calling the method.

Example? Of course!

class MovieFone
 include Dslify
end

mf = MovieFone.new
mf.movie "Can't buy me love"
mf.times 8.pm..10.pm

# Or, my preferred syntax:

class MovieFone
 def initialize(&block)
  instance_eval block if block
 end
end

MovieFone.new do
 movie "Lucas"
 times 8.pm .. 10.pm
end

One neat feature that may not be apparent is that you can support different runtime methods with the same call, but only have to evaluate it at the runtime. For instance, say you want to give either a range of times (like above) or accept an array of times you are available:

mf.times = 8.pm, 10.pm, 12.am

Simple! When you setup the search method, the argument will be of the class type you send it! Cake!

Find the gem here: http://github.com/auser/dslify/tree/master!

Using Capistrano with EC2

2008-12-17T00:00:00-08:00

Capistrano has made it somewhat difficult to use cap with ec2. It has long been known that capistrano is borked when using EC2 because it requires the public key.

The work-around has been to download the public key from EC2 before running capistrano, a very ugly solution indeed. Well, after digging into the net-ssh source, it’s obvious that the public key was only used to hold identities in a hash. Because net/ssh uses ruby’s openssl library, we can call the public key to be generated from the private key, rather than requiring it from the user!

So, without further ado, no longer do you need to snatch up the public key from your EC2 instance. The fork is located at: http://github.com/auser/net-ssh/tree/master

Git-style binaries

2008-12-16T00:00:00-08:00

In this version of PoolParty internals, we’ll check out the git-style binaries used in PoolParty. Note, for this episode, I may move a little fast as there is a lot to cover.

Git, if you don’t already know is a super sleek SCM (Source control management system). It gives you hundreds of binaries using the idiom: git action options command… One big benefit to this is that all the binaries are self-contained, but function together.

With PoolParty, it makes sense to contain the binaries, both to enable quick development and to target the action the command-line method will be doing. This enables us to do cloud action, such as cloud start and cloud ssh. How do we do this? Let’s dive in!

Opening poolparty/bin/cloud, we see the following

#!/usr/bin/env ruby
require "poolparty"
require "poolpartycl"

name = ARGV.select {|arg| arg 
    if Binary.available_binaries_for("cloud").include?(arg) }.first

# Hiding methods here that strip out actions that aren't included
# in the PoolParty list of available command-line actions
# for brevity

# If no command is passed in, show help
new_args.push("-h") unless name

o = PoolParty::Optioner.new(new_args, 
  {:extra_help => "Cloud actions\n#{Binary.list_binaries_for("cloud")}", 
  :abstract => true}) do |opts, optioner|  
  opts.on('-n cloudname', '--name name', 'Address this cloud')    
    { |c| optioner.cloudname c }
end

program_name = "#{File.basename($0)}-#{name}"
program_location = File.join(Binary.binary_directory, program_name)

command_line = "#{program_location}"

# Run it from the command-line first
if Binary.available_binaries_for("cloud").include?(name)
  system command_line, *ARGV
else
  puts "Unknown poolparty binary: #{name}"
end

Phew, that’s a lot. It’s super simple, the first few lines are ensuring that the action is contained in the list of available actions.

These are gathered just by looking in the directory with the binaries that start with “cloud,” in this case:

Dir["#{binary_directory}/#{ty}-*"].map {|a| File.basename(a.gsub(/#{ty}-/, '')) }.sort

Moving on, the Optioner is a custom optsparser class that sits inside of PoolParty that gives PoolParty the ability to extend it’s own option parser. Not much to the class really, but I urge you to check out the class if you are interested on knowing how it works.

Finally, we are going to call the command! We have to rebuild the arguments sent in such that the arguments are parsed properly by the next binary that will consume them. We can use the splat operator so that when the optsparser reads them again, it’s as though the arguments were entered on the command-line.

Phew, that was quite a bit! As always, I urge you to check out PoolParty if you haven’t already. It’s pretty spiffy and super fun!

Talk to your cloud directly with ruberl

2008-12-04T00:00:00-08:00

Want to communicate with your cloud quickly?

Easy! First, download the gem ruberl

gem install auser-ruberl

Now you just have to set host and port and you can communicate with your gen_tcp server (or, in PoolParty’s case, the messenger) with either messenger_send! or messenger_cast! like so:

require "ruberl"

class Test < Ruberl::Base
end

@t = Test.new("75.101.162.232", 7050)
loop do
  nodes = @t.messenger_send!("get_current_nodes")  
  load = @t.messenger_send!("get_current_load cpu").to_f
  puts "Load on nodes #{nodes}: #{load}"
  sleep(10)
end

Alternatively, you can just set the attr_accessors host and port within your class and do the same thing, for instance

class Test < Ruberl::Base
 ... # some more methods
 def get_load
  @host = "host"
  @port = "port"
  messenger_send!("get_current_load cpu")
 end
 ...
end

Stay tuned for more quick tips, internals and discussions!

Inherited resourcing and the power of subclassing

2008-11-27T00:00:00-08:00

In this edition of PoolParty internals, I’m going to show you how the basic resourcing is done. Virtual resourcing is out of the scope of this blog post, but we’ll come back to that on another blog post.

If you are not familiar with PoolParty, I urge you to check it out

From within PoolParty, resources are a cornerstone. Most all the resources implemented in PoolParty have the same basic structure of each other, but require different parameters. For instance, a file resource can have content associated with it, while an exec cannot. However, they both behave like each other in the context of usage.

Most of the resource classes in PoolParty are tiny. For instance, the Host resource is 4 lines long without a method defined.

module PoolParty
  module Resources        

    class Host < Resource
      default_options({
        :name => "$hostname",
        :ip => "$ipaddress"
      })
    end

  end
end

When the following code is in the pool.spec, the a new host resource is created (we’ll see later that it means a new instance of the Host class is instantiated) and stored in the containing class.

	has_host(:name => "orca", :ip => "67.125.84.125")
	# Note that this is the same as calling
	host(:name => "orca", :ip => "67.125.84.125")

Almost all of the resources implementation are that small. So how do we do this?

	def self.inherited(subclass)
	end

Because we already use the method_missing to create methods from on the cloud to keep our DSL clean, we will use inherited to create the methods we want on our containing class as well as to build a list of the available resources. Let’s look at the method currently implemented in PoolParty:

def self.inherited(subclass)
  subclass = subclass.to_s.split("::")[-1] if subclass.to_s.index("::")
  lowercase_class_name = subclass.to_s.underscore

  # Add add resource method to the Resources module
  unless PoolParty::Resources.respond_to?(lowercase_class_name.to_sym)
    method =<<-EOE
      def #{lowercase_class_name}(opts={}, parent=self, &amp;blk)
        add_resource(:#{lowercase_class_name}, opts, parent, &amp;blk)
      end
      def get_#{lowercase_class_name}(name)
        get_resource(:#{lowercase_class_name}, name) if in_a_resource_store?(:#{lowercase_class_name}, name)
      end
    EOE
    PoolParty::Resources.module_eval method
    PoolParty::Resources.add_has_and_does_not_have_methods_for(lowercase_class_name.to_sym)

    available_resources << subclass
  end
end
def self.add_has_and_does_not_have_methods_for(type=:file)
  module_eval <<-EOE
    def has_#{type}(opts={}, parent=self, &amp;block)
      #{type}(#{type == :exec ? "opts" : "{:is_present => ''}.merge(opts)"}, parent, &amp;block)
    end
    def does_not_have_#{type}(opts={}, parent=self, &amp;block)
      #{type}(#{type == :exec ? "opts" : "{:is_absent => ''}.merge(opts)"}, parent, &amp;block)
    end
  EOE
end

I’m going to make this quick, but feel free to stop me and ask me if something is confusing…

Whenever a class is subclassed, the parent class receives a callback hook called “inherited.” This method is called on any class that gets subclassed, for instance:

class A
  def self.inherited(sublcass)
    puts "inherited from #{sublcass}"
  end
end
class B < A
end

# inherited from B

This way, in order to add a resource to PoolParty, all we have to do is subclass the PoolParty::Resources::Resource class.

I’m making sure that we only have to run this method once, so that both the method doesn’t get uselessly overwritten and second that it’s nicer on GC. Second, we are adding two methods at the time to the resources (note, that clouds, pools, and plugins all include the Resources module, thus all these methods are allowed within those contexts.

Finally, we add the methods onto the class, add the add_has_and_does_not_have_methods_for methods to the class (this is where the has_ and does_not_have methods get added).

One thing you’ll note from this is that every single resource created can be called 3 ways, resource_name, has_resource_name and does_not_have_resource_name. Finally, we add it to the available_resources array just so we keep track of these.

I hope that works for you and that you can find a use for self.inherited.

Update

Stay tuned for a special inside PoolParty about rspec extensions… pretty hot stuff, IMHO.

Daemonizing ruby

2008-11-21T00:00:00-08:00

This edition of PoolParty internals is going to show you how to run a block of ruby. Considering that you know a little bit of ruby, I’ll be going a little faster in this post.

There are a few commands right now in PoolParty that allow you to daemonize the process with the -d flag. (Future post on extending the OptionParser). This will place the block of code in the binary into the daemonized mode.

From within PoolParty, this command looks like:

      
def daemonize(&block)
        vputs "Daemonizing..."

        pid = fork do
          Signal.trap('HUP', 'IGNORE') # Don't die upon logout
          File.open("/dev/null", "r+") do |devnull|
            $stdout.reopen(devnull)
            $stderr.reopen(devnull)
            $stdin.reopen(devnull) unless @use_stdin
          end
          block.call if block
        end
        Process.detach(pid)
        pid
      end

I’ll walk through this snippet of ruby with you.

First off, vputs is a PoolParty internal to print verbosely, so we can ignore that for now. Basically, it calls put if verbose is set on the containing object. Until the post referenced above is written, just trust that it puts if verbose is set.

Second, we use Kernel.fork to create a subprocess from within ruby. Second, we tell the subprocess not to quit if it receives the linux signal ‘HUP.’ This way, we can safely ignore the process and assume it will continue to run even if we do log out or our OS sends it a HUP signal.

Then we’ll pipe the output to /dev/null so that we don’t see any output on the daemonzied process. It would be odd to daemonize a process and see output after we’ve forked it off.

Finally, we’ll use the block that we’ll send it when calling daemonize before detaching the process from this process. Detaching the process will set the process apart from this parent process and assume that it is it’s own parent process. This way, our process won’t receive any errors if the forked one does.

You can call daemonize with a block, like so

  daemonize do
    sum = (1..12000000).inject(0) do |sum,i|
     sum + i
    end

    File.open("/tmp/sum", "w+") {|f| f &lt;&lt; sum}
  end

That completes this edition of inside PoolParty. Stay tuned for the next one!

Code has been attached daemonize.

Update

def daemonize(&block)
  trap("CHLD") {Process.wait(-1, Process::WNOHANG)}
  fork do
    Signal.trap('HUP', 'IGNORE') # Don't die upon logout
    File.open("/dev/null", "r+") do |devnull|
      $stdout.reopen(devnull)
      $stderr.reopen(devnull)
      $stdin.reopen(devnull) unless @use_stdin
    end
    block.call if block
  end
end

This sets the flag ruby to not block if no child is available and waits for any child process of the current execution block (see Process)

Also, one more note: my MacBook is officially dead and en route to Apple, so updates to PoolParty may be slow this week.

Communicating from Ruby to Erlang

2008-11-16T00:00:00-08:00

I’ve been working pretty hard on PoolParty lately. PoolParty is a combination of ruby and erlang and gives full access to the cloud using a console.

  pool console

I’m going to show you how I implemented the console, which is a container for irb and lightweight socket communication to the messenger written in erlang.

I’ll assume that you know enough about erlang to get through the code as beginning erlang is out of scope for this tutorial.

First, let’s look at the erlang messenger

-module (server).
-export (start/0).

start() ->
    {ok, LSock} = gen_tcp:listen(5678, [binary, {packet, 0}, {active, false}]),
    active(LSock).

accept(Socket) ->
  {ok, Sock} = gen_tcp:accept(Socket),
  spawn(fun() -> loop(do_recv(Sock)) end),
  accept(Socket).

do_recv(Sock) ->
    case gen_tcp:recv(Sock, 0) of
        {ok, B} ->
          [Meth|Args] = string:tokens(erlang:binary_to_list(Data), " "),
          case Meth of
            {time} ->
              {Date={Year,Month,Day},Time={Hour,Minutes,Seconds}}
                = erlang:localtime(),
              IntString =
                [ erlang:integer_to_list(A) || A &lt;- [Hour, Minutes, Seconds] ],
              ResponseString = lists:append(IntString),
              gen_tcp:send(Sock, ResponseString),
              do_recv(Sock);
            _ ->
              gen_tcp:send(Sock, B),
              do_recv(Sock);
          end;
        {error, closed} ->
          {ok, list_to_binary(Bs)}
    end.

Now we have a spawning server that spawns off responses to respond to the requests that comes into the server, handles them and sends back their responses to the caller (using gen_tcp). In our case, we’ll be calling the contrived time method that will send us back the hour minutes and seconds

We can start the server like this

auser ~ $ erl
Erlang (BEAM) emulator version 5.6.2 [source] [async-threads:0] [kernel-poll:false]

Eshell V5.6.2  (abort with ^G)
1> c(server.erl).
2> server:start(7050).

Now that we have the server running, let’s get our ruby side talking. Our gen_tcp-based server above starts a listener at the port 5678, sounds like something we can hook up using ruby’s TCPSocket class. Which is exactly what we will do:

require "socket"
  module Messenger
    def initialize(host="localhost", port=5678)
      @host = host;@port = port
    end
    def with_socket(&amp;block)
      begin
        socket = TCPSocket.open(@host, @port)
        out = yield(socket)
        socket.close
        out
      rescue Exception => e
      end
    end
    def call!(msg="time")
      with_socket do |sock|
        sock.send(msg, 0)
        @str = sock.recv(2000)
      end
      @str
    end
    def cast!(msg="list")
      with_socket do |sock|
        sock.send(msg, 0)
      end
    end
  end

So we can call call! on any class that includes Messenger and it will send the data using a socket we open with the with_socket method. Upon sending the message, we also immediately open a socket to listen for the response. If we just want to send a message to the erlang messenger, then we can just call cast! with the message we want to send

class Test
    include Messenger
  end

@t = Test.new
@t.call!("time")

That about wraps up this edition of inside PoolParty, check out Labs at CitrusByte for more information about PoolParty.

Have fun! -Ari

PoolParty

2008-11-13T00:00:00-08:00

Obviously I have not posted in quite some time because I have been super busy.

What have you been doing Ari?

I’m glad you asked! Announcing the new PoolParty

Quickstart:

  gem install poolparty
  pool spec [name]
  cloud start

That’s it?

What did I just do?

Installed poolparty, generated a pool spec(ification) file, and launched an autoscaling, auto-maintained, highly monitored and load-balanced cloud of instances ready and booting up.

Yup! That’s it. The last 4 months of work between 11pm and 3am and the last few weeks full-time thanks to CitrusByte.

I’ve learned a lot in the past few months writing PoolParty, including erlang and a ton about DSLs. The code base is super and clean and only looks to get better and better. For more information, check out the site at: http://poolpartyrb.com