How to use Memcached with PyOWM

This is just a little demonstration on how you can quickly change the basic cache provider provided by the PyOWM library. For this purpose we’ll use Memcached, which – simply put – is a key/value in-memory data store: this turns it into a perfect caching mechanism. I’ve never used Memcached before writing this post: this shall be a good moment for me to get to know it. This demo requires that you work on a Linux env, as Memcached originally is shipped for Unix-like systems via packet distribution systems (but can nevertheless be compiled from source).

I’ll use Ubuntu, with Memcached 1.4.6 and PyOWM 0.4.0. Let’s dive into it.

First we install Memcached and the relative Python bindings:

sudo apt-get install memcached python-memcache

Then we install PyOWM library and check the installation:

sudo pip install pyowm
ls /usr/local/lib/python2.6/dist-packages # check installation

(in my distro, Python packages are installed by pip in the /usr/local/lib/python2.6/dist-packages folder: change accordingly to yours) In order to “plug” Memcached support into PyOWM we are going to leverage the installed Python bindings by creating an adapter class that can conform the SW interface that PyOWM expects into the Memcached API for getting/setting cache elements. Fortunately, the Memcached API is very close to the PyOWM expected interface (which is stated into the pyowm.abstractions.owmcache.OWMCache class), so we have chances that our adapter will be simple enough. Let’s name it “memcachedadapter.py“: you can put it anywhere, provided that this anywhere is “seen” by the Python intepreter: in example, you can put it into a folder listed into the PYTHONPATH variable or you can place it directly into the PyOWM install folder. I did the latter:

cd /usr/local/lib/python2.6/dist-packages/pyowm
sudo vim memcachedadapter.py

The module will contain the MemcachedAdapter class:

#!/usr/bin/env python

class MemcachedAdapter():
  """
  Realizes the pyowm.abstractions.owmcache.OWMCache interface
  adapting a memcache.Client object
  """
  __ITEM_LIFETIME_MILLISECONDS = 1000*60*10 # Ten minutes

  def __init__(self, hostname="127.0.0.1",
    port="11211", item_lifetime=__ITEM_LIFETIME_MILLISECONDS):
    from memcache import Client
    self._memcached = Client([hostname+":"+port])
    self._item_lifetime = item_lifetime

  def get(self, request_url):
    return self._memcached.get(request_url)

  def set(self, request_url, response_json):
    self._memcached.set(request_url, response_json, self._item_lifetime)

I wrote this adapter in 5 minutes, so please don’t blame me for errors 😉 It can surely be improved. Now what is left to do is to tell the PyOWM library how to use the adapter: this is done via configuration. The library uses OWMCache concrete instance which is created into a configuration file and injected into the code; a separate configuration file exist for the code supporting each Open Weather Map web API version. Currently only API version 2.5 is supported, so we’ll put our adapter into the pyowm.webapi25.configuration25.py file, commenting out the default cache object:

# Cache provider to be used
# cache = NullCache()  # <-- comment this line
from memcachedadapter import MemcachedAdapter
cache = MemcachedAdapter("127.0.0.1", "11211")

As you can see, we are adapting a local Memcached instance listening on the default 11211 port, but you can change this configuration as needed. Now let’s try it out – let’s start Memcached and use the PyOWM library:

memcached &
python

in example:

>>> from pyowm import OWM
>>> owm = OWM()
>>> f = owm.daily_forecast("London,uk")  # This first call to the API is not cached, obviously
>>> g = owm.daily_forecast("London,uk")  # This second call is cached

Time saving should be at a glance.

In a similar way it is possible to write adapters for plugging other cache/storage providers (Redis, MongoDB, etc..) into the PyOWM library.

EDIT: this post stimulated me to write more adapters, you can find them here.

How to convert VMWare virtual machines to Virtual Box

Where I started from

This is my situation: I’m on a Windows7 x86 host, and I have an old Ubuntu 10.04 virtual machine with VMWare Tools installed on it.

My need is to turn it into an OVF appliance, so that I can run it on Virtual Box, no matter where what architecture it will be run on.

Steps

This is what I’ve done:

  • I made sure (e.g: using VMWare Player or Workstation) that the virtual appliance is powered off;
  • Opened a Command Prompt, moved to the VMWare Player/Workstation installation dir and executed the OVF conversion tool. Be aware that this conversion may take some time, depending on how big is your VMX appliance. I did it as follows (replace the paths as needed):
cd "C:\Program Files\VMware\VMware Workstation\OVFTool"
ovftool.exe "C:\Users\claudio\Documents\Virtual Machines\ubuntu1004\ubuntu1004.vmx" "C:\Users\claudio\Documents\Virtual Machines\converted-to-virtualbox\ubuntu1004.ovf"
  • When the conversion process was over, I imported the “ubuntu1004.ovf” appliance into Virtual Box by using the “File” > “Import virtual appliance ..” menu element  and leaving all the defaults;
  • Then I booted up the “ubuntu1004.ovf” appliance and performed VMWare Tools uninstallation by opening up an SSH shell and executing:
sudo /usr/bin/vmware-uninstall-tools.pl
  • Then I finished the whole process by executing the “Device” > “Install Guest Additions” menu item: a virtual CD is then mounted and I launched the installation process from a shell
cd /media/VBOXADDITIONS_4.2.12_84980
sudo bash VBoxLinuxAdditions.run

 

Not as difficult as it may seem…. Hope this helps!

The yes command: “Aye, sir!”

The dumbest “yes-man” Linux command: yes!

It has only one aim: to indefinitely repeat what you tell it. Useful if pipelined with interactive cmd line installers (where you usually have to type “y” a lot of times in order to accept the default values) and similar borying operations.

So, for example if you go with:

yes test

it keeps on printing the cmd line arguments followed by a new line character (CTRL+C to stop execution):

test
test
test
[...]

…and if you do not specify any argument, it just says:

y
y
y
y
[...]

Yes, man! 🙂

Linux paths, filenames and base directories

Sometimes – when geeking around in Linux – you need to play with things like filenames, paths and so on. Here’s a quick reference that uses readlink, basename and dirname commands.

You have a folder structure like, say:

/opt/
   |_ goofy.txt
   |_ test/
       |_ mickey.txt

Now, let’s get inside:

cd /opt

and let’s start playing:

# ABSOLUTE PATHS 
# Get absolute path of goofy.txt</span>
readlink -f goofy.txt          # Gives: /opt/goofy.txt
# ...and of mickey.txt
readlink -f test/mickey.txt    # Gives: /opt/test/mickey.txt

# FILENAMES 
# Get filename for mickey.txt without extension
basename test/mickey.txt .txt    # Gives: mickey

# BASE DIRECTORIES 
# Get base directory relatively to the current directory
dirname goofy.txt          # Gives: .
dirname test/mickey.txt    # Gives: test
#...and then absolutely
dirname "$(readlink -f goofy.txt)"        # Gives: /opt
dirname "$(readlink -f test/mickey.txt)"  # Gives: /opt/test

As you can see, it’s just…for playing 🙂