A really nice post about using the Mac clipboard in Python. Specifically, the script adds a nice bit to filter the clipboard content type. The entire site is great.
Programming
18
May 12
List Python Packages [Link]
Clark’s script prints the entire list of Python packages installed with easy_install. I don’t know what’s normal, but I feel like maybe I have too many.
16
May 12
Please Teach How To Code
I hate posting links about broadly discussed topics. I thought hard about the latest discussion1 around Jeff Atwood’s post and I still had something to say, so here it is.
To those who argue programming is an essential skill we should be teaching our children, right up there with reading, writing, and arithmetic: can you explain to me how Michael Bloomberg would be better at his day to day job of leading the largest city in the USA if he woke up one morning as a crack Java coder?
—Jeff Atwood
Jeff is very smart. I almost always agree with his posts, but I think he gets this one wrong.
Traditionally we2 have focused our primary educational institutions on Language, Math, Science and History. The presumption has been that teaching these basic topics also provides basic problem solving and critical thinking skills necessary to participate in society in a constructive way. Language provides communication skills. Math teaches us rules based problem solving. History teaches a sense of place. Science teaches general problem solving. By gaining a basic understanding of each area, a new graduate could competently perform a number of basic jobs.
Our social needs have out grown our educational curriculum. Basic empolyment now requires some fundamental understanding of computers. While we debate net neutrality and the internet as a basic human right, we are producing high school graduates that lack even the most rudimentary understanding of the tools needed to access the internet.
Later, Jeff writes:
The general populace (and its political leadership) could probably benefit most of all from a basic understanding of how computers, and the Internet, work. Being able to get around on the Internet is becoming a basic life skill, and we should be worried about fixing that first and most of all, before we start jumping all the way into code.
Please don’t advocate learning to code just for the sake of learning how to code. Or worse, because of the fat paychecks. Instead, I humbly suggest that we spend our time learning how to …
Research voraciously, and understand how the things around us work at a basic level.
Communicate effectively with other human beings.
At once, Jeff invalidates his own argument and gets to the heart of my philosophy.
I agree that teaching to code for a “fat paycheck” or just to make an app are poor drivers. But teaching to code is valuable for more fundamental reasons. Our Science education in primary school is failing. On average, people leave school without the most fundamental understanding of the scientific process. At all levels of childhood we have perverted science education. Science has become CSI and chemistry sets are no more than baking soda. Programming gives us a another method for teaching the same critical thinking skills. It gives us a method that provides an immediate benefit to a 10 year old.
I can teach a 10 year old how to measure a reaction rate, but the concept will always remain abstract and the process will always seem arcane and anachronistic. However, teaching a kid to code with something like Scratch teaches them the same critcial skills and shows them how it relates to their world.
I’m a realist. I would love for everyone to learn science. I would really love it if people understood the data supporting evolution, global thermodynamics and vaccines. The reality is that we are not teaching those things well and students are not learning the skills necessary to teach themselves.
As a scientist and a hacky computer programmer, I see a direct parallel between the two. They both teach the following skills:
- Abstract problem solving through visualization
- Cause and effect
- Hypothesis generation
- Hypothesis testing
- Record keeping
But science and programming teach two important lessons as well:
- The value of well executed failure
- There is no such thing as a status quo
In all of these discussions, I replaced the word “programming” with the word “science”. If I may, I will take some liberty and rephrase Jeff’s question:
To those who argue science is an essential skill we should be teaching our children, right up there with reading, writing, and arithmetic: can you explain to me how Michael Bloomberg would be better at his day to day job of leading the largest city in the USA if he woke up one morning as a crack scientist?
Yes I can.
2
May 12
Automated FTP from Dropbox with Hazel
Macstories.net linked to QuickShot 2.0. It’s an iOS app that can send photos directly to Dropbox. But I was thinking, I would prefer an app that could quickly send images to my FTP server at Macdrifter.com. So I came up with a little Hazel rule to connect Dropbox and my FTP server.
Design Considerations
I want to put these files on my FTP server so that I can use them for posts to this site. That means I need an easy way to get the URL of the image. I previously built an email rule that uploaded the file and returned the link to me in a reply email. That worked really well, but email is not where I write my posts. I wanted the URL somewhere convenient. So I decided to have the resulting links added to a URL log file in my Dropbox Notes folder. NVAlt, WriteRoom and Nebulous Notes all point to my Notes folder.
I knew I would use FTP. I really like Transmit for FTP, but it seemed a little heavy handed for Hazel automation. I decided I needed to use straight Python for the FTP and skip the application scripting.
The Hazel rule is simple. It just looks for a new file it has never matched before. All of the work is done by a Python script.
Note that the Shell path is actually invoking Python.
The Script
I learned something new with this Python script, which is generally my goal. I learned about using the Paramiko module for SFTP. Paramiko is primarily a Python module for SSH that also includes some convenient methods for SFTP. I’m not sure if it passes the Drang sniff test but I like it.
To install Paramiko, just use pip:
sudo pip install paramiko
It should also install the pycrypto dependency.
Here’s the entire Python script.
import paramiko
from datetime import date
import os
import sys
import urllib
try:
# This is how Hazel gets the incoming file path
localPath = sys.argv[1]
# Location to write the URL links
logFilePath = "/Users/weatherh/Dropbox/Notes/Linkin_Logs.txt"
# Base File Location on the FTP Host to upload the file
remotePath = "/home/macdrifter/webapps/wp/wp-content/uploads/"
fileName = os.path.basename(localPath)
# Just logging paramiko incase of issues
paramiko.util.log_to_file('/tmp/paramiko.log')
host = "my_host_name"
port = 22
# This will be the base URL path for the file link
url_Base = "http://www.macdrifter.com/wp-content/uploads/"
transport = paramiko.Transport((host, port))
password = "my_super_strong_password"
username = "my_sftp_user_name"
# Determine the current month and year to create the upload path
today = date.today()
datePath = today.strftime("%Y/%m/")
# Used to create full remote file path
remoteFilePath = remotePath + datePath + fileName
transport.connect(username = username, password = password)
sftp = paramiko.SFTPClient.from_transport(transport)
# Do the deed. Use the SFTP put command
sftp.put(localPath, remoteFilePath)
sftp.close()
transport.close()
#Open the URL log file for appending
logfile = open(logFilePath, "a")
try:
# %% encode the file name and append the URL to the log file
logfile.write(url_Base+datePath+urllib.quote(fileName)+'\n')
finally:
logfile.close()
except IOError:
pass
Obviously, you will need to provide your own connection details and file paths. This script is specific to my needs.
The Workflow
There are two ways I am using this. The first is on my Mac. Any file I drag to this Dropbox FTP folder gets uploaded and a link added my link log. It’s amazingly convenient.
The second is on iOS. The QuickShot app is cool. I configured it to upload to this Dropbox FTP folder. The file is uploaded to the server and a link silently added to my log file. I can upload any file from an application that supports Dropbox uploads. But QuickShot is pretty convenient.
I’ll also add that it is convenient to have a Dropbox folder filled with the images I have uploaded to my server. It provides a history for me to see what I was thinking while adding screen shots. I take a heck of a lot of screen shots.
Comments
There are a lot of modules imported for this script. sys is needed just to handle the connection with Hazel. os is needed just for getting the file name and writing the log file. Heck, I even import the urllib module just to encode the file name. But you know what? It works and it is fast, so I don’t care. CPU cycles are cheap. Isn’t that the point of these scripting languages?
It’s never a great idea to hard code login credentials. I might change this in the future to use a config file, but I’m not too concerned with the kind of stuff I use this for.
The error handling in the script is pretty lean. Other than the try and except blocks there is none. I should add more. I probably will not.
The FTP put command will overwrite any file with the same name in the same location. That’s by design. If I want to edit a file on my Mac, I can just open it in Acorn or optimize it with ImageOptim and Hazel will reprocess it when I save. The new version is uploaded but the URL link remains the same.
This will work with any file type. That’s cool but a bit scary. If I accidentally drop-in something private, then it will be uploaded to my public FTP host automatically. I’m aware of the potential problems. I’m a big boy. I can handle it.
Future Plans
I’d like to add some automatic image processing. I definitely want to include ImageOptim processing but I would also like to resize any image larger than 650px. It would save a lot of trouble with iPad images. I might need some different rules to provide multiple options for image resolution.
I’m thinking about adding the links to my pasteboard on my Mac. That way I would have the log file and also a running list in my pasteboard.
27
Apr 12
JavaScript Closures and Variables [Link]
A good introduction to JavaScript closures and variable scope. This stuff always gets me. Also, I rather like anchovies on pizza.
27
Apr 12
Python Outputty [Link]
I don’t remember how I stumbled across this, but Outputty is a Python module for transforming structured tabular data from one format to another. For example, converting CSV to an HTML table. Just look at this tutorial. Nice.
13
Apr 12
Yoda Perl [Link]
So, yeah, this never gets old.
I guess I do still have some uses for Perl.
By way of many wasted moments on the Internet.
11
Apr 12
Renaming Finder Items [Link]
A very nice workflow at Macgrunt.com for reliably renaming a set of known files with new names.
By way of Clark’s Tech Blog
10
Apr 12
Python and P-lists [Link]
I learned five things from this 554 word post by Dr. Drang about determining the default browser on OS X.
- There’s a Python library for reading plists
- There’s a Python library for creating temp files that auto-delete
- The InternetConfig tool is still around in OS X 10.7
- Dr. Drang uses AppleScript like I do, as a wrapper for Shell and Python
- I invested in learning the right scripting language for the distopian future of AppleScript
29
Mar 12
Javascript Code Syntax Highlighting [Link]
The Rainbow JS library looks very interesting. I use the CodeColorer WordPress plugin it works well and has broad language support. But I really like the idea of using JavaScript instead. It would be independent of WordPress and Rainbow is easily customizable. I really like that it can use its own CSS.
If you notice intermittent oddness with code on Macdrifter, it’s because I’m playing around with Rainbow.
By way of Jonathan Christopher
16
Mar 12
FTP Files to WordPress by Email
Yes, another hack to make this site easier for me to manage.
I post a lot from Simplenote now. While this is very efficient for making link-posts, it makes it rather challenging to create a post with images. Images are stored on the Macdrifter host in the standard WordPress content directories but I need a URL to create the image link reference in Simplenote.
I have used GoodReader in the past to upload an image to the server, but I have to manually create the URL for the image. That’s no fun.
So I created this insult to everyone that appreciates clean code.2 The script is attached to a mail rule on my primary Mac. When I send an email message with an attached image1 and a subject prefix “iii”, the script is triggered. It saves the image to a specific directory on my Mac and then uploads it to Macdrifter.com by FTP using Transmit3. Upon successful upload, it sends a reply email with the URL to the image.
The Abomination
(*
Author: Gabe Weatherhead
date: 2012-03-16_090533
Script for FTP upload to WordPress via Apple Mail rule processing.
This work is Creative Commons-Attribution licensed. Basically, do what you want with it, but don't be dick.
http://creativecommons.org/licenses/
Mesaage subjects are expected to be prepended with "iii"
Transmit is expected a favorite to exist for the destiantion. Mine is named "Macdrifter_WP"
*)
using terms from application "Mail"
-- Uploads file using a saved favorite in Transmit
on upload_file(theFile)
try
-- Creates base URL for remote file based on standard WordPress content structure
set urlPath to "http://www.macdrifter.com/wp-content/uploads/"
set myMonth to do shell script "date +\"%m\""
set myYear to 1 * (year of (current date))
set clipContents to {}
tell application "Transmit"
-- Stored Transmit favorite
set myFav to item 1 of (favorites whose name is "Macdrifter_WP")
tell current tab of (make new document at end)
connect to myFav
-- tempMSG is used for Growl reporting of errors.
set tempMSG to "Transmit attmepting to connect"
set this_item to theFile
-- Update Error reporting
set tempMSG to (this_item as text)
-- Get the File's path
set this_info to info for this_item
set myPath to (POSIX path of this_item)
-- Update Error Reporting
set tempMSG to "Attempting to locate local file" & (myPath as text)
set myFileName to name of this_info as text
-- Transmit must open the local location that holds the file
change location of local browser to path (POSIX path of ("Macintosh HD 2:Dropbox:Todo Files:Blog:"))
-- It's just easier to encode URL safe characters with Python
set myURLFileName to do shell script "/usr/bin/python -c 'import sys, urllib; print urllib.quote(sys.argv[1])' " & ¬
quoted form of myFileName
set myURLFileName to urlPath & myYear & "/" & myMonth & "/" & myURLFileName
set end of clipContents to "
" & myURLFileName
-- Tell Transmit to do the upload
tell remote browser
upload item at path myPath to "webapps/wp/wp-content/uploads/" & myYear & "/" & ¬
myMonth & "/" with resume mode overwrite with continue after error
end tell
-- I like to have the URL put on my clipboard as well
set the clipboard to myURLFileName
close remote browser
end tell
close front window
end tell
set AppleScript's text item delimiters to "
"
set clipContents to clipContents as string
set AppleScript's text item delimiters to ""
return clipContents
on error errmesg number errn
set AppleScript's text item delimiters to ""
return errmesg & return & tempMSG & "error number: " & (errn as rich text)
end try
end upload_file
-- I prefer Growl messaging for error reporting since I also use Prowl for iOS
on growl_message(theMessage)
tell application "System Events"
set isRunning to (count of (every process whose bundle identifier is "com.Growl.GrowlHelperApp")) > 0
end tell
if isRunning then
tell application id "com.Growl.GrowlHelperApp"
set the allNotificationsList to ¬
{"FTP Mail Handler Script", "Script Success", "Script Error.app"}
set the enabledNotificationsList to ¬
{"FTP Mail Handler Script"}
register as application ¬
"FTP Mail Rule Script" all notifications allNotificationsList ¬
default notifications enabledNotificationsList ¬
icon of application "Script Editor"
notify with name "FTP Mail Handler Script" title ¬
"FTP Mail Handler" description "FTP Upload: " & ¬
theMessage application name "FTP Mail Rule Script"
end tell
end if
end growl_message
-- Mail Rule starts processing the messages that match
on perform mail action with messages theMessages for rule theRule
try
-- Just another sanity check. Only messages that begin with this prefix get processed.
set custom_prefix to "iii"
-- All of my WordPress images start in the same local location
set localPath to "/Volumes/Macintosh HD 2/Dropbox/Todo Files/Blog/"
repeat with oneMessage in theMessages
set receivedSubject to subject of oneMessage
set outgoingSubject to "Re: " & receivedSubject
set theSender to extract address from sender of oneMessage
if (receivedSubject starts with custom_prefix) then
set {mail attachment:theAttachments} to oneMessage
repeat with oneAttachment in mail attachments of oneMessage
set newFilePath to POSIX path of (("Macintosh HD 2:Dropbox:Todo Files:Blog:") & (name of oneAttachment))
--growl_message(newFilePath)
save oneAttachment in ("Macintosh HD 2:Dropbox:Todo Files:Blog:") & (name of oneAttachment)
set fileURL to upload_file(newFilePath)
growl_message(fileURL)
set mailBody to "File Uploaded to Server: " & return & return & (fileURL as rich text)
set AppleScript's text item delimiters to {""}
set theNewMessage to make new outgoing message with properties ¬
{visible:true, subject:outgoingSubject, content:mailBody, sender:"gabe@macdrifter.com"}
tell theNewMessage
make new recipient at end of to recipients with properties {address:theSender}
end tell
send theNewMessage
set AppleScript's text item delimiters to ""
end repeat
end if
end repeat
on error errmesg number errn
set AppleScript's text item delimiters to ""
growl_message(errmesg & return & return & "error number: " & (errn as rich text))
end try
end perform mail action with messages
end using terms from
Commentary
It might not be obvious, but this method requires an always-on Mac with Mail running. That’s not a problem for me.
This is written explicitly for my WordPress content structure. This is pretty standard stuff but it’s possible to use another file structure. Just change the urlPath variable appropriately.
I use Growl for notifications. About 1/3 of this script is for error handling and Growl messaging. I like this method because I also use Prowl. Any errors are displayed on my iOS devices almost immediately. I can also look over the Growl logs to see the history. Good stuff.
The script also inserts the link on my Macs clipboard. Since I use Launchbar, that means when I come back to my desk, all the links I generated remotely are right there in my history, ready to paste.
The script replies back to the same email address I sent from. I am not using the AppleScript “reply” method for Mail. It doesn’t work well. Seriously, this is why we can’t have nice AppleScripts.
Usage
This will work from any app that can send an image by email. That covers most of them, including the new iPhoto for iOS.
I wish there was an Acorn for iOS, but iPhoto does some nice stuff with images. It’s not perfect, but it is pretty good for basic image enhancements.
-
Technically, it will work with any attachment. It should handle multiple attachments as well, but I haven’t been using it that way. ↩
-
It’s bad. I’ve tried to provide comments, but that will not make the code any better. I’d clean it up, but I’m planning a total rewrite using AppleScriptObjC. That will take a long time. This works now. ↩
-
I use Transmit because it’s the best FTP client around. But it also has some nice features that make scripting easy. For example, Transmit can save my credentials with the FTP connection. The script can then login without ever needing to know the credentials. When I change my FTP credentials, I only need to change them in Transmit, not 25 different scripts. ↩
18
Feb 12
Explaining List Comprehensions [Link]
A great explanation of list comprehensions at DZone. I’ve used them several times and the linked Python.org explanation was what I used to understand them. I’m not a mathematician or a Programmer but list comprehensions are pretty cool.
9
Feb 12
More Python and Markdown Goodness
I’m currently looking at incorporating some more markdown functionality in a few personal Python centric projects I have. There is some interesting stuff in this space.
Sundown is a markdown parser for Python and many other languages. Specifically, Misaka is the Python implementation of Sundown.
Pyhame is a static html generator for markdown with support for code highlighting. Installation looks simple.
Ever2Simple is a Python module for migrataing Evernote documents to Simplenote with conversion to markdown. This looks very interesting.
Leaf is billed as a Python library for parsing HTML. But there is a nice feature to convert html to markdown.
9
Feb 12
Python-Markdown Extensions
From Waylan Limberg at achinghead.com. A tutorial for implementing a new Python-Markdown extension. This provides an easy way to add custom markdown processing with only a few lines of code. In the examples, he implements strike through, underline and alternatives for bold and italic.
I’m using a different Python module for my Simplenote to WordPress blogging engine but this one looks very interesting.
Here’s his Github for the project
Edit: I’m an idiot. This is the module I’m using. I know what I’m doing this weekend.
8
Feb 12
Jeff Atwood Impact
Jeff will definitely be missed.
Fever is great at measuring what’s important. Here’s what Fever is telling me about Jeff’s community:
He’s smart and I completely respect his decision. I wish him well and thank him for such a great contribution to his “little corner of the Internet”. Well done Sir.
