First foray into open source with Proselint

For 2016, I challenged myself to contribute to 10 different open source projects on github.

I started looking for a project to work on by scanning through the list of trending github projects. I eventually decided to work on Proselint. The purpose of this project is to develop a tool that could analyze writing - it's a linter for prose.

I want to be a better writer, so a tool to improve my writing would be good to know about and to use. It would also allow me to 'dog-food' software. I like projects where I'm also the user.

Looking through the github page, there were a lot of existing small issues, many of which were tagged 'Easy'. I eventually chose to fix an issue where I would add a check that looks for paragraphs that start with the word 'But'. It seemed like a good balance of an easy problem that I could solve, but not a completely trivial code update.

I eventually made the code change, got my pull request accepted, and became an open source contributer. It was a really small update, but even still, I learned and was reminded of a number of things.

• pip is a really cool package management program. I should incorporate it into my projects more.
• regex is powerful. I shouldn't be afraid of it.
• nosetests is a great way to run tests. I should use this more.
• I need to figure out a good way to organize files/code better. Having a good structure makes things much easier to find and update.
• Normally, gvim is my go to text editor, but for this new project Atom with the vim plugin might be a better option. It's much easier to navigate across files and folders and it has some auto-complete to move things along.

This was my first open source contribution adventure, and it was fun, inspirational (for me), and educational. For my next projects, I will look for a variety of projects (different languages, different applications, etc ...). I think the variation will teach me a lot.

As they say, it's certainly better to give than to receive.

Trying out Flask

I have been building a new web application, called Done Notes.

To build the application, I have been playing around with some new toys:

• Flask is a web framework for Python. It's supposedly a micro-framework, but it seems to do a lot for you. This makes it easy to do things, but hard to debug.

• Postgresql is an open souced sql database. I have been playing with no-sql data stores, so it's very comfortable to return to using a relational, sql-based database.

• Heroku is a cloud application platform. I haven't really had a need to debug things on the cloud or stressed the resource limitations. So far, it has been just fine.

• Alembic is a database migration tool. This tool connects my data models with database migrations. It generates upgrade and downgrade statements that are real sql statements (or very close to them). It makes version controlling your database changes easy to do.

I think it's nice to work with new tools once in a while. It can be frustrating, but it often helps you think differently.

Into the unknown with Google App Engine

As mentioned before, I am setting up a cloud back-end for my mobile app.

This has involved writing code in a few different languages.  

• Java - for the android application.
• Python - for writing the Google Endpoints API.
• Javascript/Html/Css - for writing a web client for the endpoints API. 

Working from the tutorials, I have been able to progress without too much issue.  However, I ran into a little bit of confusion when trying to pass around a datetime object.  Specifically, I wanted to:

1. Start with a date / time value. 
2. Pass that to my endpoints api. 
3. Store it in the Google App Engine datastore.

I couldn't find solid documentation on how this works, so it took me a while to figure out. Here are the steps:

1. In javascript code pass in your date time for some field like so:
   { some_field : "2011-12-16T10:00:00.000" }
   The format is Year-Month-Date"T"Hour:Minute:Second.FractionalSeconds
   When I tested, passing in a time zone offset threw an error.
2. In python, when you construct your date time field use:
   from protorpc import message_types
   some_field = message_types.DateTimeField(1, required = True)
   # This is a little bit different from passing other fields which use protorpc.messages
3. Once you have the datetime field, you can use the type ndb.DateTimeProperty in your model that you will insert.

This all works by passing in a string, which gets parsed and converted to a datetime object.  If you didn't want to use protorpc conversion to a datetime object, you can pass in a datetime string that you'd have to encode and decode.

Update - After I set up an Android client to send data to my endpoints api, I got more errors about the datetime having a bad timezone format.  After some trial and error then googling for solutions, I decided that dealing with datetime data wasn't worth the hassle.  I changed the datastore to only save "Long" values.  Then I updated the API to send and receive long values.  Finally, when I get to the UI, I convert the long into date time format for display.  This made things much simpler and cleaner.

Mobile App + Cloud Backend

I wanted to set up a cloud back end for my Android application for a few reasons:

• To practice and learn about cloud and web development.
• To improve my Android app by backing up user data and to provide additional analysis based on a more full history for a given user and aggregated data from multiple users.

As a beginner web programmer, I didn't know where to begin.  Despite some criticisms, I turned to Google Cloud Endpoints in Python to provide a solution.  This works with Google App Engine and gives you a bunch of things: such as user authentication, data storage, ways to generate libraries for different clients (such as Android or the web) and scalability (although my user base is pretty small for now).

I started with this sample tictactoe application.  After a few minutes I got the sample application to work on my local machine.  When I moved the application to the cloud as a Google App Engine application, I got errors about "Origin mismatch".  The resolution was to make sure you registered any clients with the correct url.  This includes matching the protocol type as well - ie use "https".

After a few hours (spread out over a few weeks) I was able to create some new request/response objects which are used in the messages passed to and from the back end to a client.  I think I have a decent feel for how things work. In my head, the cloud back end goes through the following steps.

1. A Web Client (or Android) creates JSON (or java) message object. 
2. Your generated backend libraries convert these message objects into some request object.
3. The request object is then sent to one of your backend apis/methods.
4. (In my case) I take the request object and put it into the Google App Engine datastore.  In your case, you can do whatever you want.
5. A response message is created and sent back to the original web or mobile client.
6. Your generated backend libraries convert the response message back into a message. 
7. Your client can then do whatever they want with the response.

I haven't actually implemented or tested anything in the Android client, so we'll see if my thinking about how this all changes. 

Google app engine python search tutorial

After completing an introductory tutorial for Google App Engine using python,  I was able to create the first version of an app engine project, englishtokoreanforum.appspot.com.  I modified and extended the tutorial a bit to set up a few other handlers and data types.

I then went on to the next Google App Engine tutorial, which would teach me how to set up searching over an index that I set up for this app.  Unfortunately, when I first tried it, the application in the tutorial would run, because of this issue.  (It has since been fixed after I upgraded to the next version of the GoogleAppEngineLauncher).  

Along with setting up the search index and functionality, this tutorial is nice because it isn't as trivial as the first tutorial.  It shows how someone might want to organize all the files that might go into a real project.  The different files types in this tutorial are *.html, *.py, *.yaml, *js, *css. It shows how to separate data files, test files and the main application.  This is invaluable for a beginner, and something that isn't often emphasized.

Google App Engine - Python

I already spent some time building a Google App engine product in Java. Java was a natural choice because because I was already working with Java (and Google) as I was building an Android App.  While I was/am making some progress it has been slower than I had hoped.  So, for my next Google App Engine App, I am going to give Python a try.

Again, similar to Java, I'm not very familiar with python, so this will be another learning experience.  I started by following this tutorial, https://developers.google.com/appengine/docs/python/gettingstartedpython27/introduction, after which I was able to do many fundamental web actions (displaying a site, reading/writing to a datastore and responding to events).

My initial feeling is that it is great to develop without eclipse.  I believe that eclipse can be a great and powerful tool, but I have not mastered it yet.  Learning a new programming language and a new framework and a new IDE has been slow for me.

Just using a terminal to start a development server, mvim to code, and a browser to view is refreshingly simple.  Eclipse worked like magic, but for a beginner like me, it felt like black magic where I didn't what was going on.  One feature that I know I will miss when coding with mvim is code completion.

So, without further ado, here's the app I created.
http://englishtokoreanforum.appspot.com/
As of right now, it is the final product of the tutorial.  In the future, I envision to make a forum where users can view and enter english to korean translations.

As a final note, I mistakenly chose this appspot domain name.  I was playing around with different titles, and hit enter by accident.  Anyway, I'll live with it for now.