Programming Blog

Guido has a nice article on designing the main function for python, that is flexible and useful when invoking from interactive python prompts.

The final template looks somewhat like this.

import sys
import getopt



class Usage(Exception):
	def __init__(self, msg):
		self.msg = msg



def main(argv=None):
	if argv is None:
		argv = sys.argv
	try:
		try:
			opts, args = getopt.getopt(argv[1:],"h",["help"])
		except getopt.error, msg:
			raise Usage(msg)
		# more code
	except Usage,err:
		print >>sys.stderr, err.msg
		print >>sys.stderr, "for help use --help"
		return 2



if __name__ == "__main__":
	sys.exit(main())

1. the main(argv=None) calls enables the script to be called from interactive python prompt. sys.argv is supplied by the environment.
2. If we use sys.exit inside the main function, then during interactive execution we will find that the Python interpreter will exit!. Using it as in the above code helps again during interactive execution.
3. The usage exception is raised at the end of the main. It catches any exception and the raised exception.

Read the article here.

The submitter is unhappy with the resolution for http://bugs.python.org/issue1205

Got to figure out the \r\n fix details of that patch.

Schnitzelmitkartoffelsalat (literally "Schnitzelwithpotatosalad", schnitzel with potato salad) is a German phrase that is used to test the operations of search engines and the methods of search engine optimization. It was first mentioned on November 15, 2002 by Steffi Abel in the newsgroup de.comm.infosystems.www.authoring.misc. It was an arbitrary phrase that had not appeared in the Google index up to that point.

Python Issues: [issue1507224] sys.path issue if sys.prefix contains a colon

Came across an excellent entry on the Memory architecture of the programs. Explaining the TEXT Segment which are basically the program code, Initialized program segment, uninitialized program segment, i.e the BSS (Block Started by Symbol, an old assembly instruction name), the stack which holds the variables and function calls which grows downwards and the heap which allocated memory dynamically, which grows upwards.

With the compiled program, the usage of size and objdump commands provides excellent finer details.

More details here, Memory Layout and the Stack

svn co http://svn.python.org/projects/python/trunk cd trunk ./configure make patch -p0 < patchfile ./python Lib/test/test_platform.py svn diff > patchfile

There is a very good reply for the concepts of fork, exec, spawn and threads. It is from the Python Tutor mailing list.

James wrote: > Thanks for the quick reply. > > Interesting. I'm a little overwhelmed with the different terminology > (fork, spawn, thread, etc.). I'm under the impression that I'm > supposed to use os.fork() or os.spawn() for something like what I'm > trying to do (start multiple instances of the I/O utility from one > Python script). >

A fork is a fundamental system call in which the OS makes a nearly identical copy of the running process. I know it's a kind of *-hole thing to say, but... if you don't know why you'd want to fork your process, you probably don't need to. Forking is usually used for disassociating yourself from your parent process to become a daemon. However, it's a basic function of the system an intrinsic in many other higher level actions.

One you don't mention is "exec", which is to replace your running process image with a new process image. You can do it from the shell, type "exec somebinary" and that binary replaces your shell process, so when the exec'd process exits, your session is terminated.

I mention that because when you combine a fork with an exec, you get a spawn. Your parent process duplicates itself, but the child process chooses to exec another process. So the child copy of the initial process is replaced by new running binary and you have a spawned process running as a child of the first.

Finally, a thread (sometimes referred to as a "lightweight process" or "lwp") is kind of like a fork, except a fork duplicates everything about the initial process (except a return code) while a thread shares state with the parent process and all its sibling threads.

The interesting thing about a python thread is that it is not an OS level thread, it is a separate execution thread, but still controlled by the python interpreter. So, while a dual processor computer can choose to execute two different processes or thread simultaneously, since there's only one python interpreter (per python process) a python thread is never run concurrently with another thread in the same python process. It's more of a conceptual thing,

> However, from what I gather from the documentation, os.fork() is > going to fork the python Python script that calls the original fork > (so fork won't directly start off the programs that I need). How > would I go about forking + then executing an application? Isn't this > what spawn does? Or should I stick with fork + exec*? >

However, what you are trying to do, i.e. spawn multiple concurrent child processes, could actually take advantage of a multi processor system using python threads. If you created multiple threads, many of which spawned an independent subprocess, those subprocesses could be executed concurrently by different processors, while their status was still being coordinated via the python thread model.

Just give it a go knowing that it is an efficient design, and drop us a line if you have more questions or any problems.

Sincerely, e.

> Lots to learn, I guess. ;) >

Always. ;-) When you think there's nothing else to learn, you've already become obsolete.

> .james > > On Sep 19, 2007, at 10:19 PM, Kent Johnson wrote: > > >> James wrote: >> >>> Hi. :) >>> I have a question regarding threading in Python. I'm trying to >>> write a wrapper script in Python that will spin off multiple >>> (lots!) of instances of an I/O benchmark/testing utility. I'm >>> very interested in doing this in Python, but am unsure if this is >>> a good idea. I thought I read somewhere online that because of >>> the way Python was written, even if I spun off (forked off?) >>> multiple instances of a program, all those child processes would >>> be restricted to one CPU. Is this true? >>> >> Python *threads* are limited to a single CPU, or at least they will >> not run faster on multiple CPUs. I don't think there is any such >> restriction for forked processes. >> >> >>> I'm not sure if the utility I'm forking is CPU-intensive; it may >>> very well be. Does Python indeed have this limitation? >>> >> I would think an I/O benchmark is more likely to be I/O bound... >> >> >>> Also, should I be using os.fork() for this kind of program? >>> >> There is a fair amount of activity these days around making Python >> friendly to multi-processing. See >> http://wiki.python.org/moin/ParallelProcessing >> >> Kent >> >

_______________________________________________

This was the question from Norvig's Python IAQ. It uses python's inbuilt zip method and * operator feature.

>>> matrix = [(1,2,3),(4,5,6),(7,8,9)]
>>> print matrix
[(1, 2, 3), (4, 5, 6), (7, 8, 9)]
>>> print zip(*matrix)
[(1, 4, 7), (2, 5, 8), (3, 6, 9)]

zip method takes a sequence of arguments and returns tuple consisting of ith element of each sequence.

* is the extraction operator in the function call which can unpack the members of a list or a tuple.

Example program demonstrating the ablity of StringIO. Thanks to effbot.

#!/usr/bin/env python



"""
StringIO can be used to capture redirected output from the Python interpreter.
"""



import StringIO
import string, sys



stdout = sys.stdout # Storing the STDOUT to a variable for future restoration
sys.stdout = file = StringIO.StringIO()



print """
When the Guru administers, the users 
are hardly aware that he exists.
Next best is a sysop who is loved.
Next, one who is feared.
And worst, one who is despised.



If you don't trust the users,
you make them untrustworthy.



The Guru doesn't talk, he hacks.
When his work is done,
the users say, "Amazing:
we implemented it, all by ourselves!"
"""
try:
    sys.stdout = stdout # Restore STDOUT for print
    print string.upper(file.getvalue())
finally:
    sys.stdout = stdout # Incase some screw up happened, just restore it.

Type the following at the PYTHON PROMPT and check for what you get. :)

>>> from __future__ import braces
SyntaxError: not a chance (, line 1)