Docker HTTP proxy settings in Upstart

October 10th, 2013 2 comments

This was driving me crazy. There's some bug reports about it, but nobody has a plain and simple example. So here's mine. Enjoy.

Old:

description "Run docker"

start on filesystem or runlevel [2345]
stop on runlevel [!2345]

respawn

script
  /usr/bin/docker -d
end script

New:

description "Run docker"

start on filesystem or runlevel [2345]
stop on runlevel [!2345]

respawn

env HTTP_PROXY="http://your.address:port"
env HTTPS_PROXY="http://your.address:port"

script
  /usr/bin/docker -d
end script
Categories: Software

My first Scout plugin!

September 8th, 2013 No comments

I'm digging Scout so far, and it has almost all the plugins I would want already in their plugin directory. However, I did want to add in a Solr "healthcheck", since we've noticed some oddities with our search index.

Here is a quick-and-dirty way to get the number of results based on an empty search (i.e. the entire index) for a single Solr core on localhost. Maybe this will help somebody else out there. I suppose it could be paramterized with hostnames, search strings, etc... and it wouldn't be that hard either from what it looks like.

Enjoy.

Filename: solr.rb

class SolrResultCount < Scout::Plugin

  needs "rubygems"
  needs "json"
  needs "net/http"

  def build_report
    url = "http://localhost:8983/solr/select?q=&rows=3&fl=bundle&wt=json"
    r = Net::HTTP.get_response(URI.parse(url))
    parsed = JSON.parse(r.body)
    report(:results=>parsed["response"]["numFound"])
  end

end
Categories: Development

Setting up chrooted SFTP-only access on a Synology DiskStation

August 28th, 2013 No comments

This has been on my list to try to figure out for a long time. I wanted SFTP only access to specific accounts, and to be able to chroot them. It took me a while and various attempts, only to get wind up landing on the most basic solution, of course.

I originally tried scponly and scponlyc (which I've used in the past) and rssh, however none of them worked properly for me.

Sure enough, the openssh package from optware worked right out of the box.*

wget http://wizjos.endofinternet.net/synology/archief/syno-mvkw-bootstrap_1.2-7_arm-ds111.xsh
ipkg install openssh openssh-sftp-server

Then edit /opt/etc/openssh/sshd_config, and put in:

Match User username
        ChrootDirectory /some/directory
        ForceCommand internal-sftp

Also edit the user account in /etc/passwd, change the home dir to the /some/directory, and give it "/bin/sh" for a shell.

Viola... when sshd is restarted next time it will just work.

The guys at optware made a neat startup script that will start their sshd on boot. So nothing to do there.

Make sure to disable synology's built-in ssh (Control Panel > Terminal) or you'll probably be hitting the wrong one!

If you are concerned about privileges, the way that Synology runs its units isn't very UNIX permission friendly (most files are world writable on the filesystem, and the expectation is the daemons will properly control the access.) I wound up creating a little cron job that chmods and chowns files to keep the secondary account I've created to be a "read only" account to that directory.

* As always with my tips, YMMV - this worked fine on my Atom-based DS2411+ unit. but when I tried the same setup on a DS213, it didn't seem to work. No idea why, there aren't much diagnostics or logs provided to use. Sorry.

UPDATE: After running this on the "working" NAS unit for a bit, it stopped working. The culprit was the ChrootDirectory became owned by the user, not by root:root. Changing it back (chown root:root /some/directory) fixes that. So it looks like OpenSSH wants that in place for the chroot stuff to work. That could have been the issue mentioned in the previous paragraph (couldn't test it anymore)

Categories: Software

Tackling the "to-do" list problem

August 8th, 2013 No comments

I have a to-do list (surprise!) - actually I have a couple. Possibly even a few... I've even got an item on one of them to consolidate the lists together.

It is so easy to keep adding new items, and it isn't as easy to mark them off. Tasks change status or become mini "projects" with multiple steps (get car oil changed = schedule oil change appointment, which leads to an appointment being booked, which will be the item until it can be marked complete.) At some point I really want to visit the whole "to-do item status" concept as well.

Anyway, this is how the inner workings of my engineer brain function. Some tasks are insanely simple, but mixed in with more complex ones that require perquisite tasks, specific times of the day (business hours for example), specific people, or specific locations that may or may not be possible to get to easily.

I've struggled with trying to tame the never-ending lists. Last week I had a night open and plans to be really productive and knock some things off my list - which wound up not happening, but other "productive" tasks did get taken care of. Those were not planned, but still helpful. Someone said "what a productive day" and I felt like it wasn't the "productive" I actually wanted.

Working in the world of software/web development, this kind of stuff has parallels in the engineering world. I guess you can call that "technical debt" - new stuff is coming up and old stuff isn't being taken care of.

That won't work for me. I need to be making progress, I've got a lot of tasks that need to be finished. That's why they made the list to begin with.

While I've always had the desire to finish these things, I haven't had the proper personal accountability for actually checking things off the list(s) - I love marking things done, but I don't make it a regular habit of checking them enough.

Enter Beeminder - a personal accountability system for reaching goals. A co-worker introduced me to the site. I started thinking about the usual suspects - like losing weight and thought about some of the things he had put in - like making sure he spends X hours a day being productive on personal projects. What I liked most is the work spent on the system to make it statistically sound, measurable (a goal needs to be measurable), capable of automating, and with appropriate notifications to remind me to log whatever data isn't being supplied automatically via other devices or services (your scale can talk to it, for example.)

I started thinking about other ideas to put down as goals. One of them was being "productive" by marking one thing off my to-do list per day. However, that can vary. Some items are easier than others, and what if I have a day where other "productive" tasks come up that take away from that list (as a lot of days do...) - so I began thinking more about it.

I came up with the idea that each task should have some sort of amount of effort associated to it. The effort is derived from the amount of work the task requires and/or the amount of coordination due to location, time constraints, people constraints, etc. For example, getting my passport - that has been on my list for over a year. I paid for it, filed the paperwork, just need the photo and actual submission done. I tried a few times last year but never had all my ducks in a row, and then it fell by the wayside (I had no real need for it and got busy with other things.)

Being a obsessed with completing things like I am, it's still on my list, and I want to get it done. I won't be surprised if my payment is now forfeited due to government accounting needing to close the books each year or something like that. Nevertheless, it's something I should have anyway, it's staying on the list. That would get a value of 1.0 - the highest a task can have, in my world. It requires finding a place that is open, a place that does photos, possibly trying to haggle with them to honor my payment made last year, etc. - a lot of possible effort there, along with time constraints (business hours, sort of) and location (specific locations handle specific things.)

Once I thought about it in that way, I could put in a Beeminder task of "finish at least 1.0 units of effort per day" or something. I started thinking of other tasks on my list, and it looked like that would be an easy way to try to knock things out. It almost becomes a game.

Then I started expanding the idea further. Those random unplanned tasks that come up, such as helping my Parents move? That took a lot of physical work and time. I can't really be taking care of other items if I am busy helping them out. I don't want to be penalized for it. So we can introduce the concept of "bonus units" - I could say "I didn't do anything on the list, but that was definitely worth 1.0" - and feel like the day is fulfilled still.

I believe every day people should do something to advance their life (or someone elses), and this way of tracking makes it easier to be accountable.

Extending it even more, random daily tasks that may or may not be done, based on laziness, distractions, whatever - those can have some value too. Maybe you don't go to the store that often - so going to the store is 0.2, or getting a haircut is 0.2. What about laundry? Those things do count. They are productive. Most of us probably wouldn't think of giving ourselves some credit for them when introducing something as detailed as this.

Those people who prefer the stick vs. the carrot could even put in negative units. Maybe you want to stop watching so much TV, so you deduct 0.1 units per hour. Combine that with earning 0.2 units per hour of gym time, and you are doing the equivalent of calorie counting but with productivity.

The gamification makes it kinda fun, and it might make it easier to adopt as part of a daily routine. Tying it to a system such as Beeminder could be useful too, if you like their method of being punished for non-compliance. Some of us need that, and I may wind up using their site for tracking each day's units.

I tried to come up with a name for a unit, the best I could come up with was "Personal Productivity Unit" (or PPU, since it needs an acronym) - but it was an idea that has been brewing for a while, and I think I have come up with enough structure now to give it a shot in real life and see how it fares.

Finally, I am sure I am not the first one to think of something like this, there are probably books written on this, but I haven't seen anything myself and it makes my engineering brain happy to be able to weigh tasks and set a goal based on that. If anyone else has any more ideas on the subject, I'm all ears!

Categories: Lifestyle

Upstart script for Apache Solr

July 11th, 2013 No comments

(Apologies for not sharing my technical thoughts for six months now!)

I was trying to figure out the best way to launch Apache Solr on Ubuntu - and was having issues finding a nice clean way to do it. I decided after some misc init scripts, that I should look at Upstart. Thankfully someone (mentioned below) already had a working script available to start from.

Assumes:

  • Solr will be running as user "solr" group "solr"
  • Solr's root where start.jar is located is /home/solr
  • This works well using Solr 4.3.1, Ubuntu 12.04.2 LTS (precise)

Put this in /etc/init/solr.conf:

start on runlevel [2345]
stop on runlevel [!2345]

kill timeout 30
respawn

setuid solr
setgid solr

script
   chdir /home/solr
   exec /usr/bin/java -jar start.jar
end script

Big thanks to Eric Wilson's blog for the initial script. I tweaked it for my specific user/group/location desires.

Categories: Software

Remapping remote keys for OpenELEC/XBMC

January 10th, 2013 No comments

This past week, I set my Parents up with a Raspberry Pi-based OpenELEC unit, which I built for them to replace an extremely dated Buffalo product. (So far I have been pretty surprised as to what this little RPi can do!)

After getting a stable setup going, the last piece of it was getting a remote that they could manage. I was nervous at first because of compatibility concerns and "bleeding edge" software. However, I picked up an "Adesso ARC-1100 Media Center Remote" - which I had seen as being compatible on the XBMC forums, and out of the box, it works great. Truly plug and play.

I wanted to pick up something reasonably priced and locally (just in case I had to return it, I didn't want to hassle with RMA processes.)

The main issue with the remote is that some buttons are a bit confusing - which is tolerable, but the main offender is the power button. It works out of the box, as one would expect, except that once you power down the OpenELEC/RPi combination, you can't power it back up again (at least not with a USB-powered IR receiver...)

To solve that issue I looked into figuring out how to do some key remapping, to make the power button do something different, or nothing at all. The documentation seems straightforward but made me dizzy for a moment (and is slightly incorrect where remote.xml is), but on the first try I had success. After that, I decided to mess around and see what other buttons I could "disable" essentially (and I decided to see if <null> could be used to map the key to nothing - it seems to work)

Here is the output of that work, it looks simple enough. Put this under /storage/.xbmc/userdata/Lircmap.xml (or via the userdata SMB share)


  
    KEY_POWER 
    KEY_DELETE 
    KEY_SUBTITLE 
    KEY_VOLUMEDOWN 
    KEY_VOLUMEUP 
  

Sure enough, it all works - now I have a remote solution for them that won't let them get into trouble, and confidence now on how to map keys in the future.

After playing with XBMC on a Pivos AIOS DS, Intel NUC, and OpenELEC on an RPi, I have to say I am happy to see the extensive community involvement in the XBMC project and its derivatives.

Now, I just wish skinning it was as easy...

Categories: Toys

Shout out to CrashPlan!

August 24th, 2012 No comments

While I am typically a BackBlaze fan boy ("we'll always be unlimited" and so cost-effective) I somehow stumbled upon CrashPlan. Which when looking at it and seeing it's a Java-based client initially scared me, but the support for Linux and other OSes got me interested. Not only can I stick it on servers and home Linux boxes (and I have now...) but they even give you tips on how to use an SSH tunnel to connect to their local service. So I can launch a desktop application on my Windows machine and connect to my CrashPlan backup daemon on my server at SoftLayer. Neat.

It is supposed to be totally unlimited as well, and they only charge $12.99 or something for 2-10 computers, vs. a per-computer model from BackBlaze. Also, they don't list support for file shares, but it had no problem backing up one of my samba mounts. (Please don't fix that if it's a bug!)

So while I still consider BackBlaze to be more efficient and easier to use (just "set it and forget it" I will say that CrashPlan has a lot more options, is an opt-in policy (akin to Mozy, etc.) instead of an opt-out by default policy (BackBlaze) and it makes it really easy to list the entire filesystem, and select/deselect at any level of it.

The other interesting/neat thing is you can set it up to backup to friends machines, local storage, attached drives or their CrashPlan Central cloud (which is what they charge the monthly for.)

Since BackBlaze isn't playing in the Linux space yet, and has special ways to check if a filesystem is "local" and such, it looks like I will be using the best of both systems for now. The Java UI does feel a bit "Java-ey" but the price, features and performance of the actual network backups seem well worth it.

So +1 to CrashPlan!

Categories: Software

One of my favorite parts about conferences is the schwag.

August 20th, 2012 No comments

I'm a sucker for a free t-shirt. This year was a record - 11 shirts from OSCON 2012, and I missed going to the OpenStack anniversary event where I could have picked up another one. The one disappointment about all of this though is probably half of these shirts are unusable - they're either so thin they couldn't even make "undershirt" status, or such an odd fit (too skinny and long, or too wide and too short.)

So I plead with you, companies, please invest just that little bit more to get higher quality stock. It can't be that much more, and you'll have a greater chance that people will wear those shirts sporting your logo, and not because they picked it up at the local Goodwill. :)

On another note, I apologize for not posting anything in almost seven months. I've been busy and haven't felt like wordsmithing anything I've been up to, or otherwise haven't had anything useful to share.

Categories: Uncategorized

PHP 5.4 stuff I'm jazzed about...

January 28th, 2012 No comments

I'm always excited to read updates to the NEWS file for each PHP version (yes, I am that big of a PHP fan boy) - and PHP 5.4 has quite a handful of noteworthy changes. I started cutting them out and decided to publish my "this is interesting" or "this is awesome" list...

This is as of PHP 5.4.0 RC6 (NEWS). Note that I didn't put in anything related to OO, as I despise the obsession with OO now in PHP and do my best to live without it.

Misc. notable changes:

  • Added built-in web server that is intended for testing purpose. (Moriyoshi)
  • Changed default value of "default_charset" php.ini option from ISO-8859-1 to UTF-8. (Rasmus)
  • Added array dereferencing support. (Felipe)
  • Added header_register_callback() which is invoked immediately prior to the sending of headers and after default headers have been added. (Scott)
  • Changed http_response_code() to be able to set a response code. (Kalle)
  • Added new json_encode() option JSON_PRETTY_PRINT. FR #44331. (Adam)
  • Changed silent conversion of array to string to produce a notice. (Patrick)
  • Removed support for putenv("TZ=..") for setting the timezone. (Derick)
  • Removed the timezone guessing algorithm in case the timezone isn't set with date.timezone or date_default_timezone_set(). Instead of a guessed timezone, "UTC" is now used instead. (Derick)
  • ext/mysql, mysqli and pdo_mysql now use mysqlnd by default. (Johannes) )I think this is in 5.3 though too?)
  • Expose session status via new function, session_status (FR #52982) (Arpad)
  • Added support for storing upload progress feedback in session data. (Arnaud)

PHP-FPM related:

  • Remove EXPERIMENTAL flag. (fat)
  • Added partial syslog support (on error_log only). FR #52052. (fat)
  • Lowered default value for Process Manager. FR #54098. (fat)
  • Enhance security by limiting access to user defined extensions. FR #55181. (fat)
  • Added process.max to control the number of process FPM can fork. FR #55166. (fat)
  • Dropped restriction of not setting the same value multiple times, the last one holds. (giovanni at giacobbi dot net, fat)

Removed legacy features:

  • Safe mode and all related ini options. (Kalle)
  • register_globals and register_long_arrays ini options. (Kalle)
  • import_request_variables(). (Kalle)
  • allow_call_time_pass_reference. (Pierrick)
  • Session bug compatibility mode (session.bug_compat_42 and session.bug_compat_warn ini options). (Kalle)
  • session_is_registered(), session_register() and session_unregister() functions. (Kalle)
  • y2k_compliance ini option. (Kalle)
  • Removed magic_quotes_gpc, magic_quotes_runtime and magic_quotes_sybase ini options. get_magic_quotes_gpc, get_magic_quotes_runtime are kept but always return false, set_magic_quotes_runtime raises an E_CORE_ERROR. (Pierrick, Pierre)
Categories: PHP, PHP-FPM

PHP.reboot - are you kidding me?

October 9th, 2011 No comments

Something just cropped up today on HN about a "reboot of PHP" - being a PHP fanboy, I decided to go look. I've had my own ideas on what I'd change (or rather, just clean up, optimize, and purge) from PHP.

The project is here: https://code.google.com/p/phpreboot/

Why is this an issue? Well, for one, it's NOT A REBOOT OF PHP. It's a frickin' Java-based re-implementation of some PHP ideas and function names with a completely different syntax, and at the end of the day, it has 99.9% nothing in common with PHP.

Why do people develop in PHP? Because it is PHP. Stop trying to make PHP more like Java, more JSON-y, etc. Why did it become the world's most popular language? Besides for being easy to pick up (too easy, sometimes, which leads to a bunch of garbage and unsecure code), it got there because of what it is.

This "php.reboot" project is just trying to use PHP's popularity and function names to get people to check it out. PHP doesn't have XML/JSON/SQL style constructs (although the new array syntax sure does look like an attempt to emulate JSON, cough), it has structure - that's what "$" and ";" are for - denoting specific constructs in the language. If people don't want to develop using "$" or ";" go switch to another language that doesn't, that is already established.

I am tired of seeing blog posts and other items pop up every so often "why PHP wants to be more like Java" or "10 things PHP can learn from Ruby" - if you're trying to adapt PHP to another language, just use the other language. Period.

Some of the ideas in this project might be neat, or good; but in the end, it's not a "reboot of PHP" and stop labeling it as one.

Categories: PHP