1. LIONlab
  2. LIONoso
  3. learningfromdata
  4. lionoso_homework4

Learning From Data - Homework 4 - A solution in LIONoso

Courtesy of Giovanni Pellegrini

Exercises in week 4 are about the Bias-Variance dilemma, the Generalization error and the VC dimension.
Our script gives a solution to the exercises number 4,5 and 6 about Bias-Variance

Before proceeding, make sure you are familiar with instructions for using Python scripts to LIONoso .

Analyzing Bias-Variance in LIONoso

You can download the script Exercise4-BiasVariance.py, containing our solution to the exercise.
Please read the notes for Windows users below if you are using Windows.

Load the script by dragging a Parametric table element into the workbench, and by specifying the filename of your script.

In the above figure we just loaded the script (Exercise4-BiasVariance.py).
Specify the values of the input parameter in the left panel, where you can the number of tests (more tests for an accurate average) and the number of datasets to estimate the average function ḡ().

By clicking the "Compute" button, the script is launched and a table containing the results of each experiment is produced. For every test the table contains: the coefficient a of h(x) = a*x, variance and bias.

To compute the average of each parameter open a Bubblechart from the output table (right click on the table generated, select "New panel"->"Bubble") and drag the parameter onto the y axis. Then select the "Advanced properties" tab in the left panel and select "Show polynomial fit" with 0 degree. A red line appears on the plot showing the average value.

Repeat for the other two columns, drag one of them onto the y axis and fit.
To zoom on a specific range, just select the part in the chart.


Results

Our results of 100 tests with 100 datasets for each test are:

ḡ():
Average: 1.43

Bias
Average: 0.27

Variance:
Average: 0.23

Notes for Windows users

While on most UNIX-based systems (such as Linux and Mac OS X) it is possible to declare the script interpreter in the top line of the script, Windows bases the choice of the interpreter on the filename extension. There can be two types of problems:

  1. The interpreter is installed, but it did not register the file extension (as it happens, e.g., with R)
  2. A specialized application “stole” the file extension and is executed in place of the interpreter (as it happens, e.g., with Canopy, which appropriates the .py extension of Python)
In these cases, it is possible to execute the script from within LIONoso by providing a “wrapper shell script”. In the Python case, use a text editor (e.g., Notepad) to create the file Exercise7-Perceptron-b.bat containing the following text: 
        @echo off
        C:\Python27\python.exe Exercise4-BiasVariance.py %*
where C:\Python27\python.exe must be replaced by the path of the python.exe executable in your system. Next, import this file in the Parametric table.