<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
     xmlns:content="http://purl.org/rss/1.0/modules/content/"
     xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
     xmlns:atom="http://www.w3.org/2005/Atom"
     xmlns:dc="http://purl.org/dc/elements/1.1/"
     xmlns:wfw="http://wellformedweb.org/CommentAPI/"
     >
  <channel>
    <title>Piece Of Py(thon)</title>
    <link>http://pieceofpy.com</link>
    <description>A static blog engine/compiler</description>
    <pubDate>Fri, 13 Jan 2012 15:29:42 GMT</pubDate>
    <generator>Blogofile</generator>
    <sy:updatePeriod>hourly</sy:updatePeriod>
    <sy:updateFrequency>1</sy:updateFrequency>
    <item>
      <title>SQLalchemy Cleanup Challenge</title>
      <link>http://pieceofpy.com/2011/10/13/sqlalchemy-cleanup-challenge</link>
      <pubDate>Thu, 13 Oct 2011 14:40:00 EDT</pubDate>
      <category><![CDATA[python]]></category>
      <category><![CDATA[sqlalchemy]]></category>
      <guid>http://pieceofpy.com/2011/10/13/sqlalchemy-cleanup-challenge</guid>
      <description>SQLalchemy Cleanup Challenge</description>
      <content:encoded><![CDATA[<p id="p1">Yesterday I found myself writing some very interesting SQLalchemy. The problem is I have a date column in
PostgreSQL that is stored as epoch time, so it is just an Interger column. I need to group by year,month and
grab the total count of status='A' groups for that year,month combination.</p>
<p id="p2">Here is what I came up with, can you make it cleaner? Faster? I am curious to see the different variations
people come up with.</p>
<pre class="brush: py">
        pg_date_part_month = sa.func.date_part('month',
                sa.func.to_timestamp(Group.register_time))
        pg_date_part_year = sa.func.date_part('year',
                sa.func.to_timestamp(Group.register_time))

group_month_select = ( 
            db.query(
                sa.sql.label('year', pg_date_part_year),
                sa.sql.label('month', pg_date_part_month),
                sa.sql.label('total', sa.func.count(Group.status))
            )   
            .filter_by(status='A')
            .group_by(pg_date_part_year)
            .group_by(pg_date_part_month)
            .group_by(Group.status)
            .order_by(pg_date_part_year)
            .order_by(pg_date_part_month)
        )
</pre>]]></content:encoded>
    </item>
    <item>
      <title>Using SQLAlchemy Custom Types to Convert Integers to DateTime</title>
      <link>http://pieceofpy.com/2011/10/12/sqlalchemy-custom-types-integers-datetime</link>
      <pubDate>Wed, 12 Oct 2011 20:25:00 EDT</pubDate>
      <category><![CDATA[python]]></category>
      <category><![CDATA[sqlalchemy]]></category>
      <guid>http://pieceofpy.com/2011/10/12/sqlalchemy-custom-types-integers-datetime</guid>
      <description>Using SQLAlchemy Custom Types to Convert Integers to DateTime</description>
      <content:encoded><![CDATA[<p id="p1">Today I was working on fetching out some data from an existing PostgreSQL server and generating
some BSON output that would later be imported in to MongoDB. One of the problems I ran in to was
that I needed to format the timestamps easily for each row of data.</p>
<p id="p2">Searching the internet I ran across <a href="http://threebean.wordpress.com/2011/09/01/automatically-converting-integer-timestamps-to-python-datetime-in-reflected-sqlalchemy-models/">this blog post by Ralph Bean</a>, which does just that, but at a level
that was well beyond what I needed. So taking away some inspiration from Ralph's blog post, I decided
to just go with a <a href="http://www.sqlalchemy.org/docs/core/types.html#custom-types">Custom Type</a>.</p>
<pre class="brush: py">
from time import mktime
from datetime import datetime

class IntegerDateTime(types.TypeDecorator):
    """Used for working with epoch timestamps.

Converts datetimes into epoch on the way in.
    Converts epoch timestamps to datetimes on the way out.
    """
    impl = types.INTEGER
    def process_bind_param(self, value, dialect):
        return mktime(value.timetuple())
    def process_result_value(self, value, dialect):
        return datetime.fromtimestamp(value)
</pre>

<p id="p3">Then in my reflected table, I just override the column that holds the integer representation of the
datetime I want.</p>
<pre class="brush: py">
group_table = sa.Table('groups', metadata,
    sa.Column('register_time', IntegerDateTime),
    autoload=True,
    include_columns=[
        'group_id',
    'register_time',
    'type'
    ],
)
</pre>

<p id="p4">Now when we query and begin to use our results, register_time will be a DateTime object making it
very easy to do any timedelta arithmetic or string formatting.</p>]]></content:encoded>
    </item>
    <item>
      <title>paster shell - do people know about it?</title>
      <link>http://pieceofpy.com/2010/01/21/paster-shell-do-people-know-about-it/</link>
      <pubDate>Thu, 21 Jan 2010 07:22:42 EST</pubDate>
      <category><![CDATA[sqlalchemy]]></category>
      <category><![CDATA[pylons]]></category>
      <guid>http://pieceofpy.com/?p=329</guid>
      <description>paster shell - do people know about it?</description>
      <content:encoded><![CDATA[
Today I was having a chat today about Pylons vs. Django and for the most part it was pretty diplomatic. We got to talking about the Admin interface the Django has. Which you don't have to do any extra boiler plate for, it is just there for you. With Pylons you have to use something like FormAlchemy or use Turbogears to get a similar style admin interface for your models and data.

Since we were sitting at a computer, I went ahead brought up a quick project and did a little demo of the paster shell. Sure, it involves typing and it isn't as pretty or "fast" as an admin panel, but he didn't even know it existed. One of the common things he mentioned was, "if I want to change the menus that are dynamically defined" or "if a username needs to be changed" .. and the application itself doesn't have a custom admin panel, with Pylons he had to do raw SQL.

<pre>
$paster shell pylons_config.ini

All objects from demo.lib.base are available
Additional Objects:
   mapper     -  Routes mapper object
   wsgiapp    -  This project's WSGI App instance
   app        -  paste.fixture wrapped around wsgiapp

&gt;&gt;&gt; error_user = meta.Session.query(model.User).filter_by(username='wwitzel 3').one()
&gt;&gt;&gt; # nice thing about this, is you also will get exceptions throw if more than one record exists
&gt;&gt;&gt; error_user.username
u'wwitzel 3'
&gt;&gt;&gt; error_user.username = 'wwitzel3'
&gt;&gt;&gt; meta.Session.commit()
&gt;&gt;&gt; menu_typo = meta.Session.query(model.Menu).filter_by(id=1).one()
&gt;&gt;&gt; menu_typo.value
u'Abuot'
&gt;&gt;&gt; menu_typo.value = 'About'
&gt;&gt;&gt; meta.Session.commit()
</pre>

So that is a very simple example of how one would use the paster shell to update some bad data in the database while ensuring integrity of your custom model and extension code. After I showed this to my friend he wasn't as concerned about the lack of a web interface for administration within Pylons.]]></content:encoded>
    </item>
    <item>
      <title>SQLalchemy and JSON w/ Pylons - Best Practices</title>
      <link>http://pieceofpy.com/2009/08/17/sqlalchemy-and-json-w-pylons-best-practices/</link>
      <pubDate>Mon, 17 Aug 2009 16:32:26 EDT</pubDate>
      <category><![CDATA[sqlalchemy]]></category>
      <category><![CDATA[pylons]]></category>
      <guid>http://pieceofpy.com/?p=316</guid>
      <description>SQLalchemy and JSON w/ Pylons - Best Practices</description>
      <content:encoded><![CDATA[
I asked the question I Stackoverflow and maybe it was too generic for the site, since it just got trolled with "Google keyword" by some d-bag. So I deleted it and figured I'd throw it up on my blog a see about getting some feedback from the people who read this pile about. The reason I ask this is mainly because I am preparing to do some updated screencasts for Pylons.

I've seen multiple ways referenced in official docs and I have done it a few different ways myself. I am using Pylons and I am curious what the best practices are for this common scenario?

I have used something similar to this for auto-magically making the conversion happen.
<pre class="brush: py">
# The auto-magic version
# I pulled this off a blog, forget the source.
def _sa_to_dict(obj):
    for item in obj.__dict__.items():
        if item[0][0] is '_':
            continue
        if isinstance(item[1], str):
            yield [item[0], item[1].decode()]
        else:
            yield item

def json(obj):
    if isinstance(obj, list):
        return dumps(map(dict, map(_sa_to_dict, obj)))
    else:
        return dumps(dict(_sa_to_dict(obj)))

# here is the controller
@jsonify
def index(self, format='html'):
    templates = Session.query(Template).all()
    if format == 'json':
        return json(templates)
</pre>

I have also done the version where you use the jsonify decorator and build your dictionary manually, something like this, which is ok if I need to define some custom behavior for my JSON, but as the default behavior seems excessive.

<pre class="brush: py">
@jsonify
def index(self, format='html'):
    if format == 'json':
        q = Session.query
        templates = [{'id': t.id,
                      'title': t.title,
                      'body': t.body} for t in q(Template)]
        return templates
</pre>

I've also created an inherited SA class which defines a json method and have used that on all my objects to convert them to JSON. Similar to the the fedora extensions.

Maybe I missed some obviously library out there or some obvious helper in the Pylons packages, but I feel like this is a very common task being done a dozen different ways between docs, source, and my own personal projects. Curious what others are doing / using.]]></content:encoded>
    </item>
    <item>
      <title>Jython 2.5 and snakefight for deploying Pylons w/ SQLAlchemy + Oracle.</title>
      <link>http://pieceofpy.com/2009/03/13/jython-25-and-snakefight-for-deploying-pylons-w-sqlalchemy-oracle/</link>
      <pubDate>Fri, 13 Mar 2009 17:42:34 EDT</pubDate>
      <category><![CDATA[test driven]]></category>
      <category><![CDATA[Jython]]></category>
      <category><![CDATA[sqlalchemy]]></category>
      <category><![CDATA[pylons]]></category>
      <category><![CDATA[python]]></category>
      <category><![CDATA[personal]]></category>
      <guid>http://pieceofpy.com/?p=254</guid>
      <description>Jython 2.5 and snakefight for deploying Pylons w/ SQLAlchemy + Oracle.</description>
      <content:encoded><![CDATA[
<strong>UPDATE / 13 March 2009: </strong><a href="http://pypi.python.org/pypi/snakefight">snakefight 0.3</a> now has a --include-jar option, prefer that to using my hack.

After reading <a href="http://dunderboss.blogspot.com/2009/03/deploying-pylons-apps-to-java-servlet.html">P. Jenvey's blog post about Deploying Pylons Apps to Java Servlet Containers</a> I immediately downloaded the Jython 2.5 beta and installed snakefight to give it a try. One of our services where I work is a Pylons based application. It is deployed using paster and Apache ProxyPass. Our main application is written in Java and is deployed as a war under Jetty. So if I can get my Pylons application built as a war and deployed that way, it would greatly simplify our deployment process.

[sourcecode language="bash"]
$ sudo /opt/jython25/bin/easy_install snakefight
$ /opt/jython25/bin/jython setup.py develop
$ /opt/jython25/bin/jython setup.py bdist_war --paster-config dev_r2.ini
... output of success and stuff ...
$ cp dist/project-0.6.8dev.war /opt/jetty/webapps
[/sourcecode]

Now I visit my local server and hit the project context. I get some database errors, kind of expected them. So for the time being, I'll be running this directly using Jython to speed up the debugging process. A quick googling of my DB issues turns up <a href="http://pylonshq.com/pasties/77c3184b14d6936d86d13e4e65df92d2">zxoracle for SQLalchemy</a> which uses Jython zxJDBC. I install that in to sqlalchemy/databases as zxoracle.py and give it another go. Changing the oracle:// lines in my .ini file to now read zxoracle:// Now it can't find the 3rd party Oracle libraries (ojdbc.jar).

[sourcecode]
$ cd ./dist
$ jar xf project-0.6.8dev.war
$ cd WEB-INF/lib
$ ls
# no ojdbc.jar as expected ...
$ cd ~/project
$ export CLASSPATH=/opt/jython25/jython.jar:/usr/lib/jvm/java/jre/lib/ext/ojdbc.jar
$ /opt/jython25/bin/jython /opt/jython25/bin/paster serve --reload dev_r2.ini
[/sourcecode]

Now it is looking a little better and it able to find the jar, but still a DB issue, now with SQLalchemy library. Not having a ton of time to investigate, I decide to try rolling back my SQAlachemy version for Jython. Turns out rolling back to 0.5.0 fixed the issue. I'll be investigating why it was breaking with 0.5.2 soon (tm). So now I rerun it, and get a new error.

[sourcecode lang="bash"]
AttributeError: 'ZXOracleDialect' object has no attribute 'optimize_limits'
[/sourcecode]

I decide I am just going to go in to the <a href="http://trac.pieceofpy.com/pieceofpy/browser/snakefight-java-libs">zxoracle.py and add optimize_limits = False to the ZXOracleDialect</a>. No idea what this breaks or harms, but I do it anyway and rerun the application. Success! Every thing is working now. No liking the idea of having to manually insert the Oracle jar in to the WEB-INF/lib and not really wanting to much around with environment variables, I also implemented a quick and dirty include-java-libs for snakefight, the diff for command.py is below. This allows me to pass in a : separated list of jars to include in the WEB-INF/lib. <strong>EDIT: </strong>The diff I posted isn't needed since I put it on my hg repo. <a href="http://trac.pieceofpy.com/pieceofpy/browser/snakefight-java-libs">You can grab it from here.</a>

So now I am back to building my war. Just as before.
[sourcecode lang="bash"]
$ /opt/jython25/bin/jython setup.py bdist_war --paste-config dev_r2.ini --include-java-libs /opt/jython25/extlibs/ojdbc.jar
running bdist_war
creating build/bdist.java1.6.0_12
creating build/bdist.java1.6.0_12/war
creating build/bdist.java1.6.0_12/war/WEB-INF
creating build/bdist.java1.6.0_12/war/WEB-INF/lib-python
running easy_install project
adding eggs (to WEB-INF/lib-python)
adding jars (to WEB-INF/lib)
adding WEB-INF/lib/jython.jar
adding Paste ini file (to dev_r2.ini)
adding Paste app loader (to WEB-INF/lib-python/____loadapp.py)
generating deployment descriptor
adding deployment descriptor (WEB-INF/web.xml)
created dist/project-0.6.8dev-py2.5.war
$ cp dist/project-0.6.8dev-py2.5.war /opt/jetty/webapps
$ sudo /sbin/service jetty restart
[/sourcecode]

And presto! I am in business. My pylons application is deployed under Jetty and all the <a href="http://seleniumhq.org/">selenium functional tests</a> are passing. I am sure there is probably a easier, neater, or cleaner way to do all this, but this was my first iteration through and also my first time ever deploying a WAR to a java servlet container so all in all I am happy with the results. Performance seems about the same as when running the application with paster serve, but Jetty does use a little more memory than before (expected I guess).]]></content:encoded>
    </item>
    <item>
      <title>Tags with SQLalchemy</title>
      <link>http://pieceofpy.com/2008/10/09/tags-with-sqlalchemy/</link>
      <pubDate>Thu, 09 Oct 2008 15:10:39 EDT</pubDate>
      <category><![CDATA[python]]></category>
      <category><![CDATA[sqlalchemy]]></category>
      <guid>http://pieceofpy.com/?p=164</guid>
      <description>Tags with SQLalchemy</description>
      <content:encoded><![CDATA[<p id="p1">You see lots of examples on the net for SQLalchemy. Implementing a blog, implementing a wiki, even other articles on implementing tags. Some are good, some are pretty poor, and some are just plain out of date. After some researching on best practices for implementing a Tag system with SQLalchemy I've come up with the solution you are about to read.</p>
<p id="p2">I've pulled these examples from real world production code. Just renamed them and shortened them up a little for the blog post. I pulled the naming convention right from SimpleSite example for Pylons. Here is the the table layout. Simple. A page, tag, and relation table.</p>
<p id="p3">
<pre class="brush: py">
page_table = sa.Table("page", meta.metadata,
    sa.Column("id", sa.types.Integer, sa.schema.Sequence('page_seq_id', optional=True), primary_key=True),
    sa.Column("name", sa.types.Unicode(100), nullable=False),
)

tag_table = sa.Table("tag", meta.metadata,
    sa.Column("id", sa.types.Integer, sa.schema.Sequence('taq_seq_id', optional=True), primary_key=True),
    sa.Column("name", sa.types.Unicode(50), nullable=False, unique=True),
)

pagetag_table = sa.Table("pagetag", meta.metadata,
    sa.Column("id", sa.types.Integer, sa.schema.Sequence('pagetag_seq_id', optional=True), primary_key=True),
    sa.Column("pageid", sa.types.Integer, sa.schema.ForeignKey('page.id')),
    sa.Column("tagid", sa.types.Integer, sa.schema.ForeignKey('tag.id')),
)
</pre>
</p>

<p id="p4">Now the important part, the mapper. The mapper is what is going to tell sqlalchemy what you are trying to do and how to handle and relate those ForeignKeys. It does the heavy lifting so you don't have to.</p>
<p id="p5">
<pre class="brush: py">
class Tag(object):
    pass

class Page(object):
    pass

orm.mapper(Tag, tag_table)
orm.mapper(Page, page_table, properties = {
    'tags':orm.relation(Tag, secondary=pagetag_table, cascade="all,delete-orphan"),
})
</pre>
</p>

<p id="p6">This does two things. It setups the relationship and also uses the built-in cascade rule from SQLalchemy to ensure that no orphan tags are left in the database.</p>
<p id="p7">So now we can use this model setup like so. Here, I've just started up my paster shell so I could work through some quick usage examples.</p>
<p id="p8">
<pre class="brush: py">
page = model.Page()
page.name = "Example Page"

tag = model.Tag()
name = "tag"

page.tags.append(tag)
meta.Session.save(page)
meta.Session.commit()

tag_q = meta.Session.query(model.Tag)
tags = tag_q.all()
len(tags)

# filter pages by tag(s)
page_q = meta.Session.query(model.Page)
pages = page_q.join('tags').filter_by(name="tag").all()

# delete-orphans does the work for us here...
meta.Session.delete(pages[0])
meta.Session.commit()

tags = tag_q.all()
len(tags)

# tag cloud anyone?
# see the source code linked below for a properly weighted tag cloud.
tag_q = meta.Session.query(func.count("*").label("tagcount"), model.Tag)
tag_r = tag_q.filter(model.Tag.id==model.pagetag_table.c.tagid).group_by(model.Tag.id).all()

# what about pages with related tags?
page_q = meta.Session.query(model.Page)

taglist = ["tag1", "tag2"]
tagcount = len(taglist)
page_q.join(model.Page.tags).filter(model.Tag.name.in_(taglist)).\
group_by(model.Page.id).having(func.count(model.Page.id) == tagcount).all()
</pre>
</p>

<p id="p9">Ok, now the fun part, what about all related tags? An intersection between an arbitrary number of many-to-many relationships? For that I added a static method to my tag class. Something like this.</p>
<p id="p10">
<pre class="brush: py">
class Tag(object):
    @staticmethod
    def get_related(tags=[]):
        tag_count = len(tags)

inner_q = select([pagetag_table.c.pageid])
        inner_w = inner_q.where(
            and_(pagetag_table.c.tagid == Tag.id,Tag.name.in_(tags))
        ).group_by(pagetag_table.c.pageid).having(func.count(pagetag_table.c.pageid) == tag_count).correlate(None)

outer_q = select([Tag.id, Tag.name, func.count(pagetag_table.c.shipid)])
        outer_w = outer_q.where(
            and_(pagetag_table.c.pageid.in_(inner_w),
            not_(Tag.name.in_(tags)),
            Tag.id == pagetag_table.c.tagid)
        ).group_by(pagetag_table.c.tagid)

related_tags = meta.Session.execute(outer_w).fetchall()
        return related_tags
</pre>
</p>

<p id="p11">A big thanks to <a href="http://cakephp.org/">PHP-Cake</a> and <a href="http://tagschema.com/">TagSchema</a> for the ideas, concepts, and implementation examples.</p>
<p id="p12">You can find the actual code that this blog was the basis for at:
<a href="http://trac.pieceofpy.com/pieceofpy/browser/tags-sqlalchemy">http://trac.pieceofpy.com/pieceofpy/browser/tags-sqlalchemy</a></p>]]></content:encoded>
    </item>
  </channel>
</rss>

