Slides 2020/21

For latest ones, see here

Part A

Lab A.1

Wednesday 23 Sep 2020

Links

lab site: sciprog.davidleoni.it

• Installation from sciprog, with links to relevant SoftPython stuff

• Tools and Scripts on SoftPython : describes Jupyter and other things are described more in detail

• Python basics : hopefully we will see something (try doing them at home anyway)

What I expect

• if you don’t program in Python, you don’t learn Python

• you don’t learn Python if you don’t program in Python

• to be a successful data scientist, you must know programming

• Exercise: now put the right priorities in your TODO list ;-)

Course contents

• Hands-on approach

Part A - python intro

• logic basics

• discrete structures basics

• python basics

• data cleaning

• format conversion (matrices, tables, graphs, …)

• visualization (matplotlib, graphviz)

• some analytics (with pandas)

• focus on correct code, don’t care about performance

• plus: some software engineering wisdom

Part A exams:

There will always be some practical structured exercise.

Examples:

• analizing employees of University of Trento

• visualizing intercity bus network

• extracting categories from workshops of Trentino

Sometimes, there can also be a more abstract thing with matrices / relations, (i.e. surjective relation)

DO read the exam rules

Part B - algorithms

• going from theory taught by Prof. Luca Bianco to Python 3 implementation

• performance matters

• few Python functions

Lab A.2

Monday 28 Sep 2020

Worksheets

• Finish Basics (NOTE: redownload it, was updated)

• Strings1 - Introduction

• Strings2 - Operators

• Strings3 - Methods

• (a fourth page remains to be added)

References:

• Andrea Passerini - Introduction to Python slides

• Andrea Passerini - data structures slides

Lab A.3

Wednesday 30 Set 2020

Worksheets:

• Lists1 - Introduction

• Lists2 - Operators

• Lists3 - Methods

• (a fourth page remains to be added)

References: Andrea Passerini - data structures slides

For Realtime stuff during the lab, see this repl:

https://repl.it/@DavidLeoniWork/sciprog-ds-lab-2020-21

Lab A.4

Monday 5 Oct 2020

Worksheets (now English links are finally online!)

• Tuples

• Sets

• Dictionaries1 - Introduction

• Dictionaries2 - Operators

• Dictionaries3 - Methods

• (a fourth and fifth pages remain to be added)

References:

• Andrea Passerini - data structures slides

• Real Python sets

For realtime stuff during the lab, see this repl:

https://repl.it/@DavidLeoniWork/sciprog-ds-lab-2020-21

I copy here last weekend announcment:

I added here some links to exercises: https://en.softpython.org/references.html

• Beginners: try W3Resources

• Intermediate: exercises from LeetCode and first tutorial from Software Carpentry

• Math-oriented people: try Introduction to Scientific Programming with Python by Joakim Sundnes

Lab A.5

Wednesday 7 Oct 2020

• Worksheets (now English links are finally online!)

  • Control Flow 1 - ifs

  • Control Flow 2 - for

  • Control Flow 3 - while

  • Errors and testing Note: SoftPython tutorial only talks about quick testing with assert which is fine for Part A of this course. In Part B we will use more sophisticated testing with unittest

  • References: Andrea Passerini - complex statements

For realtime stuff during the lab, see this repl:

https://repl.it/@DavidLeoniWork/sciprog-ds-lab-2020-21

Lab A.6

Monday 12 Oct 2020

Worksheets

• More on functions

• Matrices - list of lists

• Errors and testing Note: SoftPython tutorial only talks about quick testing with assert which is fine for Part A of this course. In Part B we will use more sophisticated testing with unittest

References:

• Andrea Passerini - complex statements

• Andrea Passerini - functions

For realtime stuff during the lab, see this repl:

https://repl.it/@DavidLeoniWork/sciprog-ds-lab-2020-21

Lab A.7

Wednesday 14 Oct 2020

Worksheets

• Sequences (already translated)

• Data formats (already translated)

References:

• Andrea Passerini - Modules and programs

For realtime stuff during the lab, see this repl:

https://repl.it/@DavidLeoniWork/sciprog-ds-lab-2020-21

Lab A.8

Monday 19 Oct 2020

Worksheets:

• Graph formats

  • graph matrices, adjacency lists (we will use networkx next lab)

• Binary relations (finish at home)

NOTE: In Part A we will limit ourselves to graph formats and see some theory in separate binary relations notebook, while in Part B of the course will focus on graph algorithms

For realtime stuff during the lab:

• sciprog repl

• graphviz repl

• graph drawings and challenges

How to see graphviz on repl.it

Since we’re online, I came up with this solution

NOTE: this is a shaky experiment only to share code during the lab, but you won’t need if you do the worksheets above alone.

1. Go to this NEW repl: https://repl.it/@DavidLeoniWork/softgraphviz

2. Fork the project (DO NOT copy paste files in your projects, you really need to fork !)

3. Run the script - you will have to wait some time until it loads all the packages:

NOTE: this will activate a server, which means from now on the server will be kept running and you shouldn’t need to click on Run again (unless of course something is seriously compromised)

4. Try making changes to example.py and then, instead of hitting the usual Run in repl, click the Reload button. Reload will run the script example.py, which at the end regenerates the output.dot file used in the visualization

NOTE: that after a while of inactivity, the server might stop and repl will need to reload all the packages.

Lab A.9

Wednesday 21 Oct 2020

EXAM - Save the date:

• Simulation: Monday 2 November, 14:30-16:30 (online)

• Midterm Part A: Friday 6 November, 11.30-13.30 (online) - please login 15 mins before and allocate some time after the end in case we have technical problems

Simulation attendance is recommended!: during the simulation and the exam you will connect online to a linux environment (but you do not need to install linux!), so the simulation is your chance to try the system.

You will receive official communication with details in a short time. For now please read exam guidelines from previous years

WORKSHEETS:

Today we focus on graphs as adjacency lists, networkx

• Graph formats

• Binary relations

NOTE: In Part A we will limit ourselves to graph formats and see some theory in separate binary relations notebook, while in Part B of the course will focus on graph algorithms

For realtime stuff during the lab:

• sciprog repl

• graphviz repl (to use, see instructions of previous lab)

• graph drawings and challenges

Lab A.10

Monday 26 Oct 2020

Worksheets:

Today we focus on numpy and matplotlib

• Matrices - Numpy - actually we don’t do that much of numpy, it’s more an introduction to things needed for visualization

• Visualization with matplotlib

• The AlgoRythm Opera Challenge

NOTE 1: Visualization worksheet is different from Italian version! Contains also some statistics on networks.

NOTE 2: after the lab I put on softpython a revised version of the The AlgoRythm Opera Challenge. In particular even if you were at the lab, make sure to try this new last part of the challenge, with changed variables: most probably you will discover there is some constant in your code which should be a variable instead.

References:

• Andrea Passerini Numpy and Matplotlib slides

For realtime stuff during the lab:

• sciprog repl

• graph drawings and challenges

Lab A.11

Wednesday 28 Oct 2020

Worksheets:

• Matrices - Numpy images (new!) - was proposed as a challenge during the lab, now on the website you can find a revised version with solutions. Notice images may look like an application, but actually they are matrices, only much nicer to see!

References:

• Andrea Passerini Numpy and Matplotlib slides

For realtime stuff during the lab:

• sciprog repl

• graph drawings and challenges

Lab A.12

Wednesday 4 Nov 2020

Worksheets:

• Pandas tutorial

• Pandas case study: EURES job offers

• Pandas: Italian Poets challenge

• Exercises from Edabit, I put a selection on SoftPython References page (if you don’t see it refresh the page with ctrl-R or ctrl-F5)

Note there won’t be pandas during midterm for part A, at most I will use it only to show some data. For you, it could be useful to quickly gain an understanding of dataset contents.

References:

• Andrea Passerini Pandas slides

For realtime stuff during the lab:

• sciprog repl

• graph drawings and challenges

Part B

Lab B.1

Monday 9 Nov 2020

• Remember that from now on we only use Visual Studio Code, possibly with the Live Share extension, so get familiar with it by reading installation

• Testing with unittest

• OOP (probably only first part)

Lab B.2

Wednesday 11 Nov 2020

Worksheets:

• OOP Finished ComplexNumber, try doing Multiset on your own.

• OOP Matrix Challenge: (updated after the lab, please use this version). Try solving this for next week. Tests are not provided, you are invited to collaboratively write at least some in the new ‘REPL Test Playground’. You can access the link on Moodle

Lab B.3

Monday 16 Nov 2020

Worksheets:

• Complexity: the invisible enemy

• Sorting Challenge

• Sorting Selection sort and Insertion sort

Lab B.4

Wednesday 18 Nov 2020

Worksheets:

• Complexity: the invisible enemy

• Sorting Challenge - I added Partitocracy challenge (a kind of quicksort)

• Sorting in lab we will see quicksort and mergesort, finish the rest at home

Lab B.5

Monday 23 November

• LinkedList v1

• Little LinkedList challenge

At home: try to finish whole LinkedList worksheet

Lab B.6

Wednesday 25 November

• LinkedList v2 , other linked lists exercises

• Stacks

At home: try to finish whole LinkedList and stacks worksheets

Lab B.7

Monday 30 November

• Queues (CircularQueue and ItalianQueue)

• I added some exercises about linked lists and queues on Part B References page on Sciprog

At home: try to finish whole Queues worksheet

Lab B.8

Wednesday 2 December

• Binary trees

At home: try to finish whole Binary trees worksheet

Lab B.9

Wednesday 9 December

Generic trees, at home: try to finish whole Generic trees worksheet

Also:

• during ‘extra’ lab on Monday we’ve seen tree visit using queues, I wrote something about in bintree page

• Also: I updated references sections with exercises from LeetCode about binary trees, adding also some note you might find helpful

• As a further exercise, you might try doing them also on generic trees.

Lab B.10

Monday 14 December

• Graphs algos worksheet

• Will put more material …

NOTE: although past years I didn’t give much exercises on graphs, this year you may get one on BFS, DFS, topsort, connected components …