Programming - The Answer , My Friend, is ...

on August 11, 2011

My Own Failures

1. I wrote an application in a Ruby web framework 1.x.x. Everything worked, fine. Then the 2.x.x. version came along and I upgraded and my app broke in many places. I fixed them somehow.

Then the 3.x.x. version came along and I upgraded and my app broke in many places again.

That is not the framework's problem. The real problem is I had used too many plugins.

2. I wrote an application in a PHP web framework 1.6.x. Everything worked, fine. Then the 1.7.x. version came along and I upgraded and my app broke in many places.

This is not the framework's problem. The real problem is I had used too many PEAR's.

3. Before all this, I worked for a project that migrated from EJB1 to 2 to 3 etc.
And for another project, where the project managers took years to move from Struts 1 to 2.

Other Failures

1. Netscape's spaghetti code that cost the company its battle against Microsoft.
2. Friendster's social graph that slowed the site to a crawl.
3. MySpace's "technical debt" that ruined the site.
4. Ruby on Rails framework --- Its lack of hosting (Maybe their developers don't care or it is their blind spot). And its handling of data management; if they don't know this part well enough, they should leave it alone e.g. PHP has no ORM and has been doing just fine.
5. Microsoft alone has a dozen or more data access technologies from 1998 to 2008, according to a book by Joel Spolsky. I don't have time or experience to know such stuffs.

My Own Successes

1. Porting RedBlack tree, AA tree etc from Java to Ruby is easy.
2. Porting shortest path, quick sort etc from Ruby to Python is easy.
3. Porting Regular expressions, string/text handling etc from PHP and Perl to Ruby is easy.
4. Porting higher order functions amongst Perl, Python, Ruby is easy.
5. Porting any plain domain or business object from any language to any other language is very very easy. From Java to Pascal, from Perl to PHP, from Ruby to C++, whatever.

6. Modeling or making mental representations or doing objects, classes and APIs for Law domains, Insurance domains, Education domains are very very very easy and fast.

7. Dreaming, going into trance, seeing in my mind eyes some apps end to end from the users' point of views is extremely easy and fast. "Easy" doesn't mean I do these activities well because, often, I even forgot to do them at all!

8. Writing domain objects is very easy.

9. Combining those objects is very easy.

Other Successes

1. Real programmers code everything by themselves. A paradox because they do and they don't.
2. Real programmers don't trust others' codes. A paradox because they do and they don't.

3. Excel team wrote even their own compiler, and killed off all dependencies.
4. Algorithms are very infrequent. They come once or twice in a decade.

5. Facebook is using many programming languages. So is Twitter.
6. Google founders didn't learn HTML etc on purpose.

The answer

If you can't find Apple-style appealing packaging, sell your stuffs, your wares raw, plain, uncovered!

Have fun!

From "Code and Pray" to "Crash and Fix"

August 30, 2010

Software industry is changing every hour and making progress every day. Software development methods and techniques are also getting better every year. Thus, we old folks have to stay updated and one way to achieve this is to learn from the apprentices and the interns.

This week I like to discuss different intern techniques: those in our days and those nowadays. In our intern days, one common technique was "code and pray." We would code something and run it by the complier and pray that it worked. Some practictioners also call this coincidental style of coding.

Recently I noticed that the interns at our company have developed this to a higher level. I noticed that they are now using "crash and fix" technique. It's a beauty in this particular case.

Crash and Fix GQL

As we all know, we need to create and provide an index.yaml file to Google App Engine so that it can search its data store properly.

Locally (that is on our development server), if we make a new kind of query, the Engine updates the index file. When we repeat that query some time later, the Engine is ready to do the search.

However, on the real server, if we make a new kind of query, the Engine just scans the index file. If the index necessary for the query is not there, the Engine throws exceptions.

There is no quota/limit on the local engine, but the real engine places some limit on the number of indexes.

Thus, the right thing to do, at least for us old folks, is:

1. Find typical usage patterns.


2. Make queries for just these patterns.


3. The resultant index.yaml file, developed on the local server, will be relatively small.


4. Use this small index.yaml file on the real server,

This would take time and insight. Modern interns have better ideas:

1. Deploy the app on the real server with an empty index.yaml file.


2. When a user makes a new type of query, the Engine throws excpetions. Log those exceptions somewhere.


3. Check those logs. The log tells us the specifications of the index it needs for a particular query..


4. Copy such specifications into a new index.yaml file.


5. Deploy the app on the real server again, and repeat.

This also took time but not insight. Do the modern interns have an edge over us old folks? Pay special notice to the comment "# goodpt3," "# goodpt2" etc. Those below 1 were added before those below 2 and so on. The interns will check only the topmost section, which is added the most recently. Time-saving and brain-saving, cool?

indexes:

# AUTOGENERATED

# This index.yaml is automatically updated whenever the dev_appserver
# detects that a new type of query is run.  If you want to manage the
# index.yaml file manually, remove the above marker line (the line
# saying "# AUTOGENERATED").  If you want to manage some indexes
# manually, move them above the marker line.  The index.yaml file is
# automatically uploaded to the admin console when you next deploy
# your application using appcfg.py.

...

- kind: Inventory
  properties:
  - name: expenses
  - name: locations
  - name: distance
  - name: skus
  - name: unit_space
    direction: desc
  - name: transports
...

- kind: Inventory
  properties:
  - name: gender
  - name: name
  - name: distance
  - name: skus
  - name: unit_space
    direction: desc
  - name: transports

# goodpt3
...
- kind: Inventory
  properties:
  - name: unit_space
  - name: gender
  - name: locations
  - name: phone
  - name: skus
  - name: transports
...


# goodpt2

...
 
- kind: Inventory
  properties:
  - name: unit_space
  - name: expenses
  - name: gender
  - name: locations
  - name: phone
  - name: distance
  - name: skus
  - name: transports
 
# goodpt1 - kind: Inventory
  properties:
  - name: education
  - name: transports
  - name: gender
  - name: locations
  - name: skus
  - name: unit_space
    direction: desc
...

- kind: Inventory
  properties:
  - name: transports
  - name: gender
  - name: locations
  - name: distance
  - name: skus
  - name: unit_space
    direction: desc
...

The file has almost 1200 lines and uses up 92% of the index quota. If we add new models/entities, we must make new queries and will need new indexes. Will the remaining 8% cover most of our future needs, or am I over-planning?

An Online Programming Community and their Members

Here is a verbatim invitation to the  Singapore Python User Group on Facebook.

I admire the source code of Django, Tornado, NLTK, boto, etc. As many of the world greatest coders in history advise, their logic/code is "dumb" and therefore good.

When I coded shopping carts, CRUD websites etc, my logic/code stayed "dumb" and good. But when I moved even a bit farther, e.g. FSM with Pygame, an expert system etc, my logic/code became "clever," complex and unstable.

I've thought about and studied this for years.

Our team is training to be a world leader in software -- both biz-wise and techno-wise. Anyone interested in becoming better programmers are welcome.

We are humble to learn and generous to share.

Love and Thanks to all.

I'll let you see and hear how they respond in a few days' time.