Portable Google App Engine For Windows (Enhanced)


5/12/2011: Updated to support Google App Engine Version 1.5.0.

A while back I put together the information and steps to get Google App Engine running on Windows wrapped up in a portable little bundle. Since it has been a while I thought it could use a good refresh with the latest versions of everything and make things work more like the real Google App Engine. Some of the content may seem the same as last time but there definitely will be some new ways of running things and new capabilities that were not available last time. This new way is much easier and should keep up with all the changes that newer versions of Google App Engine may bring.

Installation

Let’s gather all the files that we are going to need. I throw the version numbers on here that this was written for but yours may vary slightly. Download the following:

  • Portable Python (Download the 2.5.x release. I am using 2.5.4 at the moment.)
  • Google App Engine SDK (Download the Linux/Other Platforms release. I am using 1.4.1-2011-01-05 at the moment.)

Now that you’ve got all the files, you can grab a USB thumb drive or even just install it to a directory on your local machine. Run the Portable Python installer and specify the location you’d like to store it. I will refer to this location as the “Portable Python folder” from here on.

Extract the “google_appengine” folder from the Google App Engine SDK zip file into the Portable Python folder on your thumb drive. Side Note: In the future if you ever want to update to a newer release, just delete the “google_appengine” folder and extract a new copy from the latest zip file from Google.

Create a new folder in your Portable Python folder called “google_appengine_datastores”. This is where we are going to store all the data for our apps.

Create two new files called “dev_appserver.py.bat” and “appcfg.py.bat” in your Portable Python folder both with the following contents:

@echo off
REM Portable Google App Engine Command Line Rewriter
REM Written by David Lambert (http://www.codepenguin.com)

REM Execute the command line rewriting script
apppython.exe _cmdrewrite.py %0 %*
REM Execute the new command line batch file
_cmdrewrite.bat

Create a new file called “_cmdrewrite.py” in your Portable Python folder with the following contents:

#Portable Google App Engine Command Line Rewriter
#Written by David Lambert (http://www.codepenguin.com)

#Rewrite the command line with correct paths
import sys, os, re
new_cmd = 'apppython.exe google_appengine/%s' % sys.argv[1]
gae_version = re.search('release: "(.*?)"',open('google_appengine/VERSION','r').read()).group(1)
appname = sys.argv[-1].rstrip('/').split('/')[-1]
if sys.argv[1] == 'dev_appserver.py' and os.path.exists(appname):
  new_cmd += ' --blobstore_path=google_appengine_datastores/%s_blobstore' % (appname)
  new_cmd += ' --datastore_path=google_appengine_datastores/%s_datastore' % (appname)
  new_cmd += ' --history_path=google_appengine_datastores/%s_history' % (appname)
  if gae_version >= "1.5.0":
    new_cmd += ' --skip_sdk_update_check'
    new_cmd += ' --rdbms_sqlite_path=google_appengine_datastores/%s_rdbms' % (appname)
new_cmd += " " + " ".join(sys.argv[2:])
#Write new command to the batch file
f = open('_cmdrewrite.bat','w')
f.write(new_cmd)
f.close()

The above three files make up the command line rewriting system. It basically will take a normal command line that you see on the Google App Engine tutorials and documentation and rewrite them to work with Portable Python. This makes the Portable Google App Engine system much more useful because you can easily follow along with the real documentation and tutorials without having to adjust or rewrite your own command lines. It also makes the datastore and history files portable by storing them in the “google_appengine_datastore” folder created earlier.

Creating your Application

Now that we’ve done all the prep work let’s see how this thing actually works. Creating a new application does take a few steps but they are pretty basic and boring. The quick and dirty steps are as follows:

  1. Create the App folder.
  2. Create a bunch of source files in your App folder.
  3. Run your App!

I’ll walk you through all the above steps to give you an idea of what is going on. We are going to use the example source from Google for the actual example app content parts. It is a great idea to read over their documentation as it will give you more insight into how the Google App Engine works.

All of our applications are going to reside in their own folders inside the Portable Python folder. Create a new folder called “helloworld” in your Portable Python folder. I’ll refer to this as the App folder from now on as all the important files will go inside this folder for our first app. The next few steps come directly from the Hello World tutorial from Google.

Create a new file called “helloworld.py” in your App folder with the following contents:

print 'Content-Type: text/plain'
print ''
print 'Hello, world!'

Create a new file called “app.yaml” in your App folder with the following contents:

application: helloworld
version: 1
runtime: python
api_version: 1

handlers:
- url: /.*
  script: helloworld.py

The super important file here is the “app.yaml”. You can think of it as the master control center of your App. It has all the settings and such that tell Google App Engine what to do. Google’s documentation lays out everything for this file. We’ve got enough in ours above to get going but I definitely recommend reading their docs when you get a chance.

Running your Application

Our HelloWorld App is ready to go! Let’s get this thing up and running. We are going to need a new command interpreter window opened to our Portable Python folder. Now type the following:

dev_appserver.py helloworld

After pressing enter you should see something close to the following:

P:PortablePython_1.1_py2.5.4>dev_appserver.py helloworld
INFO     2011-01-08 15:21:00,736 appengine_rpc.py:153] Server: appengine.google.com
WARNING  2011-01-08 15:21:00,743 datastore_file_stub.py:573] Could not read data
store data from P:PortablePython_1.1_py2.5.4google_appengine_datastoreshelloworld_datastore
INFO     2011-01-08 15:21:01,075 dev_appserver_main.py:485] Running application
helloworld on port 8080: http://localhost:8080

You can ignore the warning in the middle that says “Could not read data store data”. That just means that the datastore has not been created yet. Now open a web browser and point it to the following URL: http://localhost:8080. You should now see “Hello, world!” in your browser! That is it! Your first Google App Engine application is running. When you want to stop your application press Ctrl+Pause/Break. It will prompt you “Terminate batch job (Y/N)?” so just press ‘y’ and then press enter.

Other available commands

Google App Engine’s dev_appserver.py has a lot of options and functionality to help you manage your application. The new command rewriting system should be compatible with all the options that are available. You should definitely read through the options in their documentation so that you can utilize things like clearing the datastore, changing the webserver port, using SQLite for the datastore and much more. Let’s look at a few of these commands that you will see in the tutorials.

To run the development app server and clear the current datastore use the following command:

dev_appserver.py --clear_datastore helloworld

To run the development app server using port 9999 use the following command:

dev_appserver.py --port=9999 helloworld

If you want to use SQLite for the datastore (increased performance for large amounts of content), use the following command:

dev_appserver.py --use_sqlite helloworld

You also can use the appcfg.py commands to upload your application by using the following command:

appcfg.py update helloworld

Conclusion

You now have a fully functioning Portable Google App Engine development environment! You can follow all the tutorials on the Google App Engine website with ease. If you ever need to update the Google App Engine SDK, just delete the google_appengine folder from your Portable Python folder and extract the new one from the main download site. If you have any questions or ideas on how to improve this system, please let me know!

Advertisements

8 thoughts on “Portable Google App Engine For Windows (Enhanced)”

  1. I bow before your awesomeness.

    Been looking forward to playing around with Google App Engine. Here goes my first venture into developing a non-sql powered app.

  2. First of all, Thank you 🙂
    Second: When I ran through your post I had the following output which shows two areas where items where written to the local file system.
    The first item was the appcfg_nag which ended up in MyDocuments. (mapped to P: on my machine)
    Is there any way to have that map to the USB with your rewrite script?
    The second was on the C: drive, it is a temp folder though so it is probably ok.
    ****************************************************

    I:PortablePython_1.1_py2.5.4>dev_appserver.py HelloWorld
    WARNING 2011-05-12 23:19:08,913 urlfetch_stub.py:108] No ssl package found. urlfetch will not be able to validate SSL certificates.
    INFO 2011-05-12 23:19:23,832 appengine_rpc.py:159] Server: appengine.google.com
    Allow dev_appserver to check for updates on startup? (Y/n): y
    dev_appserver will check for updates on startup. To change this setting, edit P:/.appcfg_nag
    INFO 2011-05-12 23:20:03,073 appcfg.py:440] Checking for updates to the SDK.
    INFO 2011-05-12 23:20:03,864 appcfg.py:457] The SDK is up to date.
    WARNING 2011-05-12 23:20:03,865 datastore_file_stub.py:657] Could not read datastore data from I:PortablePython_1.1_py2.5.4google_appengine_datastoresHelloWorld_datastore
    INFO 2011-05-12 23:20:03,980 rdbms_sqlite.py:58] Connecting to SQLite database ” with file ‘c:\users\acowles\appdata\local\temp\dev_appserver.rdbms’
    INFO 2011-05-12 23:20:12,309 dev_appserver_multiprocess.py:637] Running application helloworld on port 8080: http://localhost:8080

    1. Hi Art,
      I’ve updated the scripts to support the new features added to Google App Engine Version 1.5.0. The new scripts are backwards compatible. In order to be fully portable, the update checking mechanism will be disabled. That should take care of the two messages you received. Let me know if anything else doesn’t work out for you.

  3. David,
    I realize that you wrote this over a year ago, so there could be a number of things out of date, but hopefully you can help me with this. I am using Python 2.7.3.1 and the latest GAE SDK from their website. Followed all your instructions but here is the error message I get:

    E:Portable Python 2.7.3.1>dev_appserver.py helloworld
    WARNING 2012-11-15 15:47:15,742 rdbms_mysqldb.py:74] The rdbms API is not avail
    able because the MySQLdb library could not be loaded.
    Error: option –rdbms_sqlite_path not recognized

    …all of that followed by what looks like a listing of all options for the dev_appserver.py command.

    Is this because of something that has changed in GAE regarding Cloud SQL and datastores? If I don’t need a rdbms for any of my apps, can I just comment out that option from the _cmdrewrite.py file?

    1. Hi Stuart,
      They have definitely changed a lot since this post was created/updated. The -–rdbms_sqlite_path configuration parameter is no longer supported. You can comment out that line to get it up and running again but I don’t know if there will be any other unsupported features as I have not used that version. I apologize for the lack of info. Hopefully at some point I’ll have time to come back and update everything to the latest versions.

  4. Thanks, David! I got it up and running. I did comment that line out and added the option “–use_sqlite” since it gave me a warning that the datastore file stub will stop being the default in a future release.
    Working just fine for what I’m doing, but it did kick back these other two errors in case you wanted to know. Doesn’t affect what I’m doing, so I’m happy to get this portable version up and running. Many thanks!!

    E:Portable Python 2.7.3.1>dev_appserver.py guestbook
    WARNING 2012-11-15 17:44:52,515 rdbms_mysqldb.py:74] The rdbms API is not avail
    able because the MySQLdb library could not be loaded.
    INFO 2012-11-15 17:44:54,875 dev_appserver_main.py:675] Skipping update chec
    k.
    WARNING 2012-11-15 17:44:56,220 simple_search_stub.py:950] Could not read searc
    h indexes from c:docume~1smithshlocals~1tempdev_appserver.searchindexes

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s