Pyp (short for Pyed Piper) is a project to enable python text processing at the unix prompt. I suck at sed and AWK but I suck less at Python so I think this looks interesting. I wish there was a Windows equivalent to work with CMD commands. If anything could use some help, it’s the Windows CMD interface.

The module supports standard Python string processing but also includes some built in variables for accessing strings from the shell.

The “p” variable processes each line of the shell output while “pp” processes the entire block as one string. There are several interesting examples available through the module home.

Here’s what I did to install the command on my Mac:

1. Download the file and place it in the /usr/local/bin directory.
2. Set the permissions to include execute (755 should work)
3. Have fun

I learned something else interesting while reading about this module. Sony’s Imageworks has several really amazing open source projects. They were all developed to address internal needs and eventually released to the public.

I’ve been very happy with the results of my Hazel based blogging engine. 80% of my posts come through this mechanism now.¹ In a tribute to Inception this very post was uploaded by this mechanism.

As a quick recap, it allows me to blog using Simplenote or Dropbox.³ Hazel watches my Dropbox “Notes” folder for a file tagged with “post”. Once identified, Hazel copies the file to an archive, processes the MultiMarkdown and posts here at Macdrifter.com.

The system has been working flawlessly, except when I screw around with it. But something that has been frustrating is that my posts must be written with multimarkdown meta data headers. Sometimes I just want a quick post. I also just wanted a more complete system for blogging while moving around. It has been very liberating to blog directly from Simplenote.

Improvements

This new and improved version of the Python script brings the following enhancements:

1. If no title field is provided, the note title is used for the post
2. If no category is provided it defaults to “uncategorized”
3. If a new category is entered then the script creates the new category on Macdrifter.com before posting.
4. The post id is added to the text file header after it posts.
5. Added custom growl support. More on this in a future post.

There was a substantial change to the script. I really wanted to stick with the plain XMLRPC module for making posts. However this proved ever-more annoying as I tried to access more WordPress API features. I gave in and switched to the fantastic WordPress-XMLRPC module. I highly recommend this module to my future stubborn self.²

Install the module from PyPi.

I’ve also added better handling for missing meta data. Previously the script would fail when I forgot the meta data. Now it will post with the note title as the post title and use the “uncategorized” category if either piece of meta data is missing from the header.

For the custom Growl support, I’m using the GNTP module for Python(Growl Notification Transport Protocol). This can easily be disabled. For more info on GNTP see the official documentation. It is a great module and works really well. It allows me to send custom growl messages for exceptions as well.

Future

I really wanted to manipulate the OpenMeta tags from within the script. Clark has provided a great template to work off of. However, that’s a lot of over head to add one tag. I’ll wait for Hazel 3.

The next level of complexity for me, is figuring out how to handle post edits. I’d love the ability to go into Nebulous Notes and make a quick fix and have that automatically updated on the site. I’ve mapped out what that might look like, but I’m not happy with the process yet. Having the post id in the meta data is the key to making this viable. When I get more time, I’ll see what comes together.

The Meat

Here’s the guts of the process. It’s Python. It’s not good. I’m really just a hack with patience and a tolerance for failure. This may break and it could wipe out all of your notes or corrupt your WordPress site. I doubt it, but I make no guarantees. Also, if you use, just give me a hat tip. Don’t be a dick.

import markdown

from wordpress_xmlrpc import Client, WordPressPost

from wordpress_xmlrpc.methods.posts import GetRecentPosts, NewPost

from wordpress_xmlrpc import Client, WordPressCategory

from wordpress_xmlrpc.methods.categories import NewCategory, SetPostCategories, GetCategories

import gntp.notifier

import os

import sys



#Connection details for the WordPress xmlrpc connection

wpUrl = 'http://www.macdrifter.com/xmlrpc.php'

wpUser = 'account'

wpPass = 'password'

# Hold the server instance. Needed to extract categories and make post

wpServer = Client(wpUrl, wpUser, wpPass)

postCategoryList = ['uncategorized']

title = ''

uploadCategoryList = []

## For Testing. Uncomment but comment the Hazel specific comment

#filePath = "/Volumes/Macintosh HD 2/Dropbox/Macdrifter/Posts/OmniFocus Do Over.txt"

## For Hazel

filePath = sys.argv[1]



def postToWordPress(title, html, postCategoryList):

    post = WordPressPost()

    post.title = title

    post.description = html

    post.categories = postCategoryList

    post_id = wpServer.call(NewPost(post, True))

    return post_id





def allWordPressCategories():

    categoryList = wpServer.call(GetCategories())

    categoryListLower = []

    # Create a list with just lowercase.

    for cat in categoryList:

        categoryListLower.append(str(cat).lower())

    return categoryListLower





def createNewWordPressCategory(category):

    new_category = WordPressCategory()

    new_category.name = category

    new_category.cat_id = wpServer.call(NewCategory(new_category))

    return new_category.cat_id

    

def growlInitialize():

    growl = gntp.notifier.GrowlNotifier(

    applicationName = "Simple DropBlog",

    notifications = ["New Messages"],

    defaultNotifications = ["New Messages"])

    growl.register()

    return growl

    

try:

    myFile = open(filePath, "r")

    rawText = myFile.read()

    myFile.close()

    fileName = os.path.basename(filePath)

    # Start off using the file name as the post title

    title = os.path.splitext(fileName)[0]

    # Handle the odd characters. Just kill them.

    rawText = rawText.decode('utf-8')

    

    # Process with MD Extras and meta data support

    md = markdown.Markdown(extensions = ['extra', 'meta'])

    

    # Get the html text

    html = md.convert(rawText)



## If post_id exists then already posted. We're done.

## FUTURE: USE POST ID TO UPDATE EDITED POST

    if 'post_id' not in md.Meta:

        # extract the title from the meta dictionary.

        # If there is no title attribute, keep the file name as the post title

        func = getattr(md, "Meta['title']", None)

        if func:

            title = md.Meta['title']

            # title is actualy a list

            title = title[0]

        print title

        

## extract the categories but keep them as a list

        # If no categories exist then the default is uncategorized. Don't bother getting the WP list

        func = getattr(md, "Meta['category']", None)

        if func:

            postCategoryList = md.Meta['category']

            categoryList = allWordPressCategories()

                        

## Handle categories that do not already exist on WordPress blog

            for postCategory in postCategoryList:

                # Test each category term. If it does not exist, create it

                if postCategory.lower() not in categoryList:

                    new_category = createNewWordPressCategory(postCategory)

                    # For testing

                    print "new category id for : "

                    print new_category

        

## Only post if there is a title in the meta data

        if title != '' :

            newPost_id = postToWordPress(title, html, postCategoryList)

            # For testing

            print "new post id: "

            print newPost_id

            

## Append the post_id to the beginning of the file

            #Get the file contents

            myFile = open(filePath, "r")

            myFileContent = myFile.read()

            myFile.close()

            # Write the post id and append the rest of the text

            myFile = open(filePath, "w")

            myFile.write('post_id: ' + newPost_id + '\n' + myFileContent)

            myFile.close()

            postGrowl = growlInitialize()

            postMSG = newPost_id + ": " + title + " Posted to Macdrifter"

            postGrowl.notify(

                noteType = "New Messages",

                title = postMSG,

                description = "The Blog Post was successful",

                icon = None,

                sticky = False,

                priority = 1)

except Exception, e:

    postGrowl = growlInitialize()

    postMSG = "Post Failed!"

    postGrowl.notify(

        noteType = "New Messages",

        title = postMSG,

        description = e,

        icon = None,

        sticky = False,

        priority = 2)