Parse Hyperlinks - Python

 While doing my previous post (downloading content from codingbat) i got this script for parsing the url links


urls = re.findall(r'href=[\'"]p?([^\'" >]+)', line)

r - is provided to denote the string is a rawstring(we dont need to specify escape charcters)
href=[\'"] - the string must start with "HREF=" and can either have any of the characters (' - single quote, " - double quote) next to it.
p? - the next character must be a p
([^\'" >]+) - it must end with a greater than symbol which must be preceeded either by single or double quote.

Coding bat - Post Soutions

I am not sure whether this is a intellectual property right, but these are the exercises that I solved in coding bat, a site by Nick Parlante ( who was a inspiration for me to seriously look at Python )

ok I have been trying for an hour to hack into coding bat with my user name and password - using python urllib2 module, there seems to something that keeps me out of their pages.

I am just trying to download their tutorials and post it in my blog (the actual reason for creating this particular blog post), so i have downloaded this python module called mechanize which is very promising, have to try it out and see whether it works...

[EDIT:]
and it works !!! this is the download link for the web site

http://wwwsearch.sourceforge.net/mechanize/
and here is the coding that I used to get access to the web site and download the content

#/usr/bin/env python
import sys, os, re
from urllib2 import HTTPError

import mechanize
assert mechanize.__version__ >= (0, 0, 6, "a")
mech = mechanize.Browser()
mech.set_handle_robots(False)

try:
    mech.open("http://codingbat.com")
except HTTPError, e:
    sys.exit("%d: %s" % (e.code, e.msg))

mech.select_form(nr=0)
mech["uname"] = "your registered email id"
mech["pw"] =  "password"
mech.submit()

#s = mech.retrieve('http://codingbat.com','d:/aa.html')
#[EDIT]
#this page will throw up error stating the user or tag is expired, what you need to do is login to the web site and copy the link for the "DONE" page here
mech.retrieve('http://codingbat.com/done','d:/url.html')

content = open('d://url.html','r').readlines()
i=0
for line in content:
    urls = re.findall(r'href=[\'"]p?([^\'" >]+)', line)
    for url in urls:
            if url[0:4] == '/pro':
                i+=1
                print "Processing",'http:/codingbat.com'+url
                mech.retrieve(r'http://codingbat.com'+url,'d://myhack//'+str(i)+'.html')

#make sure you create a folder d:\myHack which will used to save the files.

Database access - GUYI - Part I - Server Auth Page

3:43So, we have two class that can be used to connect to the database and retrieve records based on a query (see my previous two posts).

Now let us start forming the basics of the GUI, a page/screen which will be used to get & store the connection string settings, option to load the saved connection string settings.

Following will be a summary of the functionality which needs to be done.
-> Select existing connection strings
->select database type (MY SQL/MS SQL/ORACLE)
->IP address/severname
-> database name
-> user name and password


After getting these details validate the connection setting and save them in a FLAT file.

PyGame and GUI Programming - Multiple Database Access - Part II

This is the connect_server.py file. This module will be imported in the connection py file(previous post), this is just a common piece of coding, which connects to the database and returns a connection object, being a C# programmer i can see only the DATA as a DATATABLE rather than a recordset...:-)

so a method getDataTable will retun a record set for the query passed.

and ofcourse a main method to test the code(which is executed only during command line execution of this file and not during IMPORTS)

import MySQLdb class databaseOps():     '''     This class is used to connect to the database, the connection object is accessed     with the variable db     '''     def __init__(self,hosts,username,password,database):     #open a connection if it does not have one already         self.db = MySQLdb.connect(host=hosts, user=username , passwd=password, db=database)     def get_data_table(self,query):         cursor = self.db.cursor()         try:             cursor.execute(query)             return cursor         except MySQLdb.Error, e:             print "Error %d: %s" % (e.args[0], e.args[1])             sys.exit (1) def main():     f = databaseOps('localhost','root','1234','MyFirst')     print 'Connection successful !!!' if __name__ == '__main__':     main()

PyGame and GUI Programming - Multiple Database Access - Part I

OK!! now that I have started with the actual coding part of it. Here is my folder structure

scripts/m_db_access/
server.py
connect_server.py

the server.py file will contain the scripts that i am using to identify the type of database that i will connect, I have just made a sample of MYSQL(because that is the only database that I have). here is the coding part of it

class init_connection():     '''     This class is used to initialize the connection to any kind of database,based on the database that we set in the variable "server_type" the respective module will be loaded     '''     def get_connection(self,server_type):         '''         server_type -> the type of the server to connect (MYSQL/ORACLE/MSSQL)         '''         if server_type == 'MYSQL':             import connect_server             return connect_server.databaseOps('localhost','root','xxxx','DBNAME')         else:             import somethingelse #some other import statements         def main():     '''     Test method to check the module is working or not, we can add more cases as the class grows     '''     conn_type = init_connection()     conn = conn_type.get_connection('MYSQL')     print conn.db     print 'Connection successful !!!' #this peice of code is to tell the interpreter that execute the MAIN method only# when accessed through command line and not during IMPORT if __name__ == '__main__':     main()

PyGame and GUI Programming - Multiple Database Access

Its been a while since I have started learning. What was fun in the beginning is more fun now. Python seems to be exceptionally powerful, and I have started getting the heart of it. So now I am planning to do an interface for accessing multiple databases. This will be the kind of real time project of me in Python, my way of saying "Hello World".


What?
An interface (first in command line and in GUI), that will be used to access databases.

Components ?
A server configuration screen where you can enter select Server type(Oracle/MySQL/MSSQL), enter server name, user name/password, select database.

After a database is selected, list all the tables that the database holds along with information of each object...

The easy part?
going to write in Python

The difficult part?
Everything else....

So python Enthusiast, help me with some of your thoughts, suggestions, i will try to post more code over here, once the project is in a good shape, i will codeplex it...

Happy Python !!!! 

Running Python scripts from Python

During my start up days of learning Python, i had the necessity to shuttle between python prompt and command script (terminal), because of the reason i did not know how to run python files(.py files) from Python.


this is how to do it...
import os
os.system('python somefile.py')
#simple...