Methodological framework

Why a workshop?

The purpose of a participatory workshop to design a Learning Analytics Dashboard (LAD) workshop is threefold:

For the participants, to express a need for a LAD and to initiate the development of this LAD in order to use it later.
For the participants, to discover the opportunities offered by the interaction with digital tools, and by analytics processes.
For the LAD developers, to capitalize on the needs of the users, and the corresponding LADs.

Participatory design has several interests.

It facilitates the expression of needs and allows the involvement of users.
The group dynamic allows to multiply the generation of ideas and options. It also allows more generic solutions to emerge.
It allows a better appropriation of concepts and solutions by the participants.
It is therefore also a learning vehicle for users.

What is a Learning Dashboard?

A dashboard is a grouping on a single display of different indicators on the learner, the learning process(es) and/or the learning context(s) in one or more visualizations. In our framework, this grouping is done around panels (or portions of screen)

Putting a TBA figure

This grouping is done around a given object of interest (or a coherent set). It can be used to take stock of the progress of an assignment in a course, of the participation of learners on a platform, of the mastery of a technical or self-regulation skill…

A dashboard therefore makes sense for a given user or users in a given context. These users can be learners, teachers, training managers, administrators… It must be understandable, accepted and beneficial for these users. Data visualization and annotations must be as simple as possible, and adapted to the users.

A LAD allows to understand a situation and to react

If it allows to follow an object, the TBA is also intended to note significant events, to allow the interpretation of such events, and to decide eventually on a reaction. This reaction process is a sensemaking process. The LAD is therefore a support to this cognitive process.

Putting a figure adapted from Sensemaking - derived from Lycett2016 : note, interpret, act

In a first step, a good LAD must therefore allow to note (or capture) the significant events in a first panel (a portion of the screen).

In a second step, to understand and interpret the situation, the user will base himself on hypotheses, look for signals by consulting and manipulating data (in the form of indicators). This is also called interaction. Panels** (or portions of screens) will allow to accompany this reflection aiming at giving a sense to the situation, and to decide on an action.

In a third step, once the situation is understood. Other manipulations in order to act may be necessary. Defining a group of learners according to criteria, opening a document in a list of tasks to be done, building a list of objectives… Here again, the user(s) will interact with the data, grouped in a dedicated panel.

Support for animation and information gathering

Tools for collecting needs

Different forms are proposed to encourage exchanges and to collect a maximum of information from the participants.

User sheet (or Avatar)** - to give body to a representative user. Who are they? How do they position themselves? What are their goals? Any other information that helps participants embody this user is welcome. The participants must have the impression that they know their “idealized” user well, without necessarily having him/her really exist.
The “Purpose of the LAD” sheet - allows you to specify what the LAD will be used for, what decisions will be made, and in what context it will be used.
Step sheets** - these sheets document the different steps involved in discovering, analyzing, making sense of and responding to a given situation. Do not hesitate to indicate what reasoning is at work, and how it is manifested through interactions with data.
Panel cards** - these cards correspond to pieces of LADs, relevant displays to help in the reflection, with which the user can interact. They are based on the classic forms of “wireframe” or “mockup” techniques.

Tools to support reflection

To facilitate reflection and inspiration, three types of cards are offered to participants.

“Decision” cards - these cards aim to encourage the expression of the objectives of the TBA and to choose a particular decision. They also provide information about the possibilities.
“Visualization” cards - these allow participants to easily consult the various existing visualization modes and to choose the one that seems relevant to them. Several vizualizations can be chosen.
“Data” maps - allow participants to specify the data they are interested in and associate it with the “visualization” maps. It is possible to associate several data for a visualization (comparison for example), or several visualizations for the same data (a key value for information, and a curve by clicking to detail).

Some tips for visualization

indicate here some best practices of visualization and choices of references to the [Jivet2020]

Dashboard Design Best Practices - 4 Key Principles proposed by BI Course - Dashboard Design - Laurent Brisson

See also https://dokumen.tips/education/learning-analytics-dashboards.html or Information Dashboard Design: The Effective Visual Communication of Data Stephen Few

Using the TBA

A good question to ask participants is when, how and why the TBA will be used.

The TBA must therefore be integrated into an existing or new system. Information and incentives for use can be put in place. The choices that the TBA can bring about must be possible, understood and desired by the user.

To be completed with the advice of [Wise2014]

To conclude

As proposed, this workshop has a definite interest for the user, since it allows him to better understand the TBA and to project himself in its use.

If the needs are taken up by a TBA designer, they will also be able to benefit from a new tool to be more efficient.

The initiators of this workshop and of the related tools are also collecting the various desired or existing TBAs. Do not hesitate to explain an existing TBA with these tools, and to send them the results of this analysis or of your workshops to feed the collection, and allow the sharing of new ideas.

site under construction : https://padlad.github.io/