Mirrored from Obsolete Your Idols.

So the first puppet module for postfix I tried to apply was riseup’s, from
https://labs.riseup.net/code/projects/shared-postfix, largely because I am fond of anarchists and because their README was suitably paranoid. Verdict: it may be Too Smart for me.

Here’s the manifest I wrote to test the module

node default {

  $postfix_smtp_listen    = '0.0.0.0'

  $postfix_mydestination  = '$myorigin, manjusri.org'



  include postfix::mta



  postfix::transport { 'manjusri.org':

    ensure      => present,

    destination => 'local:',

  }

}

Not very sophisticated but that’s what I wanted to test. Set up a simple receiving postfix which accepts email for manjusri.org.

Then I hit a problem, trying to use the module.

ubuntu@ip-10-248-21-143:~/puppet$ sudo puppet apply  --modulepath=/home/ubuntu/puppet/modules manifests/mx.pp

Could not find class postfix::mta for ip-10-248-21-143.us-west-2.compute.internal at /home/ubuntu/puppet/manifests/mx.pp:5 on node ip-10-248-21-143.us-west-2.compute.internal

ubuntu@ip-10-248-21-143:~/puppet$ 

That’s not good, the example given says that’s right. The module file postfix-mta.pp declares the class with the name postfix::mta. Ah. It’s the autoloader. So, simple enough, just cp modules/postfix/manifests/classes/postfix-mta.pp modules/postfix/manifests/mta.pp and let’s get on it with. Easy-peasy.

ubuntu@ip-10-248-21-143:~/puppet$ sudo puppet apply  --modulepath=/home/ubuntu/puppet/modules manifests/mx.pp

warning: Dynamic lookup of $postfix_mydestination at /home/ubuntu/puppet/modules/postfix/manifests/classes/postfix-mta.pp:39 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.

warning: Dynamic lookup of $postfix_smtp_listen at /home/ubuntu/puppet/modules/postfix/manifests/classes/postfix.pp:37 is deprecated.  Support will be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g., $classname::variable) or parameterized classes.

Could not find class common::moduledir for ip-10-248-21-143.us-west-2.compute.internal at /home/ubuntu/puppet/modules/postfix/manifests/classes/postfix.pp:75 on node ip-10-248-21-143.us-west-2.compute.internal

ubuntu@ip-10-248-21-143:~/puppet$ 

BRAAAAAAAAAAP. So now this module is wanting to invoke another module, common::moduledir. I’m not sure I want to buy into the whole suite of modules at this time so let’s table this test and flag it as Unsuccessful for the simple reason that it would require more to use it than simply making this module a submodule of my puppet repository.

Mirrored from Obsolete Your Idols.

So you’ve got some candidate offerings to pick from, but no clear winner just based on the names and the skimming of READMEs. Now what? Now we branch!

git branch test-postfix-camptocamp

git checkout test-postfix-camptocamp

git submodule add https://github.com/camptocamp/puppet-postfix.git modules/postfix

gc 'add the camptocamp postfix module as a submodule'

gp origin test-postfix-camptocamp

That is, make a git branch for testing each flavor of postfix (and why not, branches are free) and add the repo as a submodule in the branch (this example is camptocamp’s postfix module, a contender) and then commit and push it to the origin repo. If you’re testing more than one in a similar way, checkout the branch you based the testing branch on before starting a testing branch for another submodule. You may also need to rm -r .git/modules/module/postfix to avoid log confusion.

When you do that, you might get a warning like so

warning: unable to rmdir modules/postfix: Directory not empty

Which is fine and expected. Just rm modules/postfix by hand (not git rm).

Mirrored from Obsolete Your Idols.

If you know anything about me, you know that I’m almost painfully lazy, to the point that I will put hours of effort into avoiding doing work. Case in point: I wanted to build myself a Postfix server or a hundred, using Puppet. Well, that’s probably pretty easy, just grab the Package type and alter the config files, then stash them in the catalog as File types, cut, print, that’s a wrap. But oh man, that’s like 30 minutes of work. How tedious. Can’t somebody else do it? SingTFW to the rescue.

Cursory searches turned up 5 public git repositories containing Puppet modules to manage Postfix. Success! But, too much success! How could I pick the right one? I had some rough idea of what I wanted the one I used to do but maybe I’d overlooked some important feature. I’d better compare them. To start us off, here’s the list in the order the search found them.

• camptocamp on github
• example42 on github
• ghoneycutt on github
• deck on github
• riseup-shared

So I did what any self-respecting lazy person would, I fired off 5 clones in parallel and went to force  a machine to make me coffee.

Once everything was in place, I started doing tree and diff in various places to get a feel for how the repos were laid out, which features were visible in module names, which systems were supported. The very first thing I noticed is that the deck and riseup repos are virtually identical, and so I surmise lazily that the latter is built upon the former. This also meant that I can eliminate deck from consideration in favor of riseup. So now there were four.

But now it’s time to drink that coffee.

Mirrored from Obsolete Your Idols.

Yes! Of course there is! Here’s the money shot, first, so if you’re just looking for copypasta, you can move on.

node default {

  package { 'rubygems':

    ensure => installed,

  }



  exec { 'gem install rspec-puppet > /tmp/rspec.out':

    path => '/usr/bin',

    creates => '/var/lib/gems/1.8/gems/rspec-puppet-0.1.3/',

    require => Package['rubygems'],

  }

}

OK, so what’s that all about?

What it does

It installs the package named rubygems which, on at least some Debian-ish systems, provides the ‘gem’ command, beloved of Rubyists and with which I’m attempting to become better acquainted THEN it runs a command to install the rspec-puppet gem, IF the system doesn’t already have version 0.1.3 of the gem installed in a ruby version 1.8 environment. It does these things if you put it onto a system which has puppet installed and execute the command ‘sudo puppet apply <file>’ on newer puppet installs or just ‘sudo puppet <file>’ on older puppet installs.

Interestingly, this is the second version of this manifest. The first lacked the require attribute and so when I ran it on my test system, it failed to execute the gem install, first, and then installed the rubygems package. Puppet is tricky like that and if you require things happen in a certain order, you have to make that happen. You could consider that a strength or an annoyance; I didn’t think much about it until I saw a presentation by Jason Wright at Puppetcamp 2010 where he talked about how his group had designed their Puppet manifests to run once and if a second run introduced a change, it was considered an error.

Why do this?

So continuing with my simplest thing iteration of using AWS, I thought I’d try to bootstrap up a masterless Puppet environment. Why? Well, because I have no idea what my environment will look like. At this point I should probably emphasize that all of this is for my own personal AWS stuff and while I do use Puppet at my dayjob and I do use AWS at my dayjob, my usage of both is quite a bit different, there.

More specifically, we have carved out essentially a standard-if-evolving application stack and put infrastructure in place to support that but here I’m just sort of dabbling with ideas. One of those ideas is the idea of running Puppet without a Puppetmaster node or nodes.

It used to be a somewhat heretical idea to do without a Puppetmaster, though I gather PuppetLabs now fully embraces the idea, which is good. While having a central Puppetmaster has some advantages (centralized source of truth, centralized reporting), it has (for me, in AWS) some disadvantages (requires a dedicated persistent node, requires managing The Cert Situation, potentially requires special scaling considerations). There are many many use cases where running a Puppetmaster is brilliant idea.
But if you don’t run one, you trade that for needing to solve some things on your own.

The problems I’m solving right now are ‘where does a node get its catalog’ (my answer of the moment: github) and ‘how does the node know what manifests to apply’ (maotm: a manual application via puppet agent invocations). Neither of these are yet well-solved by me, I’d say, but I’m bootstrapping to something which I suspect will look more like ‘catalogs live in S3, the nodes use s3fs to get them, instance tags are used to decide which manifests to apply’. But I think I’m on the right track by writing manifests which refer to node default, because now I don’t have to worry about updating if my instance identifier changes.

Oh, and the whole point of this stupid manifest is that now I can spin up an instance to develop my Puppet catalog on and get ready to test my Puppet manifests. I haven’t quite convinced myself that I should write a test to test this manifest which sets up the ability to test a manifest (y0, d4wg) but doing so would have a sickly sweet kind of self-referentialism to it.

Mirrored from Obsolete Your Idols.

Oh, that’s easy! Roll your own AMIs!

What I did:

• got an Amazon Web Services account; that was easy, because I’m a white dude in America
• went to the Ubuntu UEC page
• clicked on one of the (EBS!) images which took me to the AWS launch instances page
• went through all the clicky bits to make sure that the instance would allow ssh traffic in the security group
• logged into the instance, and sudo apt-get update && sudo apt-get dist-upgrade’d it
• installed puppet because puppet is fucking awesome
• clicked on Create Image from the AWS instances webpage
• waited for that to finish, then terminated the running instance
• launched an instance from the AMI I’d made
• logged into the instance and did sudo apt-get install git because, yes, git is fucking awesome
• clicked on Create Image again
• waited for that to finish, then terminated the running instance

So now I’ve got an AMI for doing development for my personal projects. Which is not, at this time, a euphemism for pornography.

Mirrored from Obsolete Your Idols.

The simplest explanation is that I got too busy with work to write about work.

Then I got too busy with outside of work to write about non-work.

Then it became A Thing, that I’d gone so long without blogging.

So that’s why I’m not blogging.

Mirrored from Obsolete Your Idols.

This was the last day before a 3 day weekend and as is customary around these parts, not many people came in and the ones who did left early. I didn’t really achieve anything worth talking about, just researched some more ideas for my next few proposals.

Specifically, I looked at AWS documentation about Elastic Block Storage, I looked at new Bacula features, I looked at Chef and specifically Knife.

Then I went to drink with co-workers, former and present.

Mirrored from Obsolete Your Idols.

This day was my first day working from home at the new job, something I negotiated to get for myself. One day a week, I work from home. Unfortunately as today was the second attempt at releasing, and things took about as long as they do the first time you do something operational, this was a 14 hour day of work for me. At least I didn’t need pants to do it.

Our proposed deploy process didn’t survive its encounter with the actual server environment but that doesn’t come as a surprise; it’s still very raw and will be refined a lot, soon.

Mirrored from Obsolete Your Idols.

I had hoped to WFH on this day but we were on track to release something important so I came in to the office.

First order of the day was sharing the Windows virtualbox file with my co-workers so they could fire up IE and validate things work in that browser, too.

That led in to a playdate for something on the web, followed by a process meeting about deployment which turned into a pair programming / code review session using a laptop jacked into a projector. I recommend this if you have more than one developer in a meeting, use the projector to display their code as you talk about related topics.

Mirrored from Obsolete Your Idols.

This was my first Monday in the new office and so the focus was on all the orientation activity they only do on Mondays. I got my picture taken for a security badge (taken with a smart phone) and spent the day following up on orientation information as well as researching SSL certificate options before proposing that the company pay for a wildcard SSL certificate for use in production environments and that we self-sign a different wildcard SSL certificate for use in development and testing environments.

Then I dug in to the documentation for how to use an SSL certificate with jboss. That was more opaque than it needed to be and I’ll probably explain what I did in the next post.