Ulf Kroehne (DIPF)

Section: Hands-on Training

As a practical part of the third day, we provided hands-on training for the analysis of log-file data using LogFSM and R. The following objectives were aimed, illustrating the topics covered in the first two workshop days:

  • Preparation of log data and description of events and event-specific data
  • Illustration of the extraction of process indicators using finite state machines
  • Connecting process indicators to the assessment framework (theoretical foundation) and psychometric models (empirical foundation)

Demonstrations and hands-on tasks are combined in the workshop in such a way that participants can apply the methods under consideration to their own log data after the workshop.

Prerequisites (Installation)

  • For the practical parts of the workshop you will need a computer on which you have R and if possible RStudio installed.
  • To download the LogFSM packages and its dependencies, please make sure tis computer has internet access.
  • LogFSM, RStudio and R should all work on a Mac or Windows machine.

Video:

Online Workshop (Sessions)

Welcome

Overview & Objectives: Conceptual and terminological separation of log data (events) and derived indicators (process data)

Slides:

Video:


Session 1

Introduction & Data: Overview of the pre-processing of log data from large-scale assessments with TIMSS 2019 as focused example. Documentation of the selected items, codebook for result data and log events.

Slides:

Video:


Session 2

Method: Algorithmic extraction of indicators using finite state machines: Principles & ideas

Slides:

Video:


Session 3

Implementation: Input of the R package LogFSM with its syntax and the integration in a workflow using R.

Slides:

Video:


Session 4

TIMSS: Connecting LogFSM to (any) log file data, illustrated with selected data from TIMSS 2019. Running finite state machines to decompose the test-taking process and implement indicators based on the output data provided by LogFSM.

Slides:

Video:


Session 5

Beyond TIMSS: Illustrating the versatility of the method by running and modifying examples using LogFSM and other data from large-scale assessments (e.g., PIAAC raw log data).

Slides:

Video:


Session 6

Application: Time to use LogFSM for your own application using provided data.

Slides:

Video:


Summary and Q&A

Summary and Q&A: Summary and integration of the core ideas: Completeness of log data with regard to the intended decomposition; Empirical validation of indicators and theoretical anchoring of the used states.

Slides:

Video: