Virtual Personal Assistant in the Digital Workplace. User Research and Requirements. Results

10 min readMay 19, 2022

A Study on the Impact of Virtual Personal Assistant in Unified Communication Software on Motivation, Satisfaction, Communication, and Collaboration of Remote Employees.

During the analysis of the data collected in the experiment, two sections were distinguished: qualitative data and quantitative data.

Qualitative data was gathered through open-ended questions using MS Forms. The responses were then analyzed using an affinity map technique. The affinity map helped in organizing and categorizing the qualitative data based on common themes or patterns. This method allows for a deeper understanding of participants’ perspectives and insights regarding the VPA in UC software.

Quantitative data was obtained through self-reporting questionnaires using the UMI (User Motivation Inventory) scale. The data collected from the questionnaires was analyzed using the IBM SPSS (Statistical Package for the Social Sciences) software. For hypotheses H1, H2, and H3, an independent samples t-test was conducted to compare the mean scores of participants between the two groups (VPA with motivation and VPA without). This test helped determine whether there were significant differences in motivation, satisfaction, and usability between the two VPA versions.

For hypotheses H4 and H5, nonparametric correlations (e.g., Spearman’s rank correlation) were used to examine the relationships between VPA usability and employees’ motivation, as well as between employees’ motivation and satisfaction. These correlations helped assess the strength and direction of the relationships between variables.

By employing both qualitative and quantitative analysis techniques, the researchers aimed to gain comprehensive insights into participants’ perceptions and experiences with the VPA in UC software, and to examine the relationships between motivation, satisfaction, and usability in relation to the different VPA versions.

Qualitative results

During the research, qualitative data was collected from the participants’ responses to various questions and scenarios. After each scenario, participants were asked to provide feedback by commenting on what they liked or disliked about the VPA’s performance. They were also given the opportunity to suggest any changes or improvements that they felt should be made and provide specific details on how those changes could be implemented.

To assess the VPA’s personality and gather insights for better design, participants were asked if they had any advice on how to improve the VPA’s personality. Additionally, a final question allowed participants to provide any additional comments or remarks about the entire testing experience.

The participants’ responses were collected and analyzed using an affinity map technique. This involved organizing the responses into different categories or themes based on their content. The three main categories that emerged from the analysis were:

1. Positive feedback — Participants expressed satisfaction with the VPA’s performance and stated that they would not change anything. They generally had a positive attitude towards the VPA and its capabilities.

2. Suggestions for changes or improvements — Participants provided specific suggestions on how the VPA could be enhanced or modified to better meet their needs. These suggestions varied from minor adjustments to more significant changes in functionality or user experience.

3. Dissatisfaction or inability to complete tasks — Some participants expressed their dissatisfaction with the VPA’s performance or encountered difficulties in completing the assigned tasks. This feedback highlighted areas where the VPA may have fallen short or encountered usability issues.

Based on the insights gained from the participants’ feedback and the suggested design changes, a final VPA design was created for further testing. This design aimed to address the identified areas of improvement and provide a better user experience for future users.

Overview of dependent variables

In order to analyze the quantitative data collected during the experiment, several statistical tests were conducted using SPSS Statistics. One of the tests performed was the test for normality, which helps determine if the data follows a normal distribution.

For the purpose of comparing the means between the two groups (task-oriented VPA and motivator VPA) on the continuous dependent variables of motivation and satisfaction, an independent-samples t-test was conducted. This statistical test assesses whether there are significant differences in the means of the two groups.

The independent t-test helps to understand if the motivation and satisfaction levels of remote employees working with different VPAs (task-oriented and motivator) are dependent on the type of VPA they are using. By comparing the means of the two groups, the test can determine if there is a statistically significant difference between the groups in terms of motivation and satisfaction.

This analysis provides insights into whether the type of VPA employed has an impact on the motivation and satisfaction levels of remote employees using UC software.

Normality tests of dependent variables

To confirm the normal distribution of the dependent variables (motivation, satisfaction, and usability), a normality test was conducted using SPSS Statistics. The results of the normality test indicated that the p-values for all dependent variables were greater than 0.05, indicating that the data followed a normal distribution.

Here is an example table to illustrate the normality test results:

As you can see, the p-values for all the dependent variables (Motivation, Satisfaction, and Usability) are greater than 0.05, indicating that the data is normally distributed.

This confirms that the assumptions of normality are met for the independent-samples t-test conducted to compare the means between the two groups (task-oriented VPA and motivator VPA) on the dependent variables.

Motivation

Based on the results of the independent-samples t-test, there was no significant difference in employees’ motivation when using the UC software with VPA between the task-oriented VPA and the motivator VPA. The t-value was 0.948, and the associated p-value was 0.349.

Since the p-value (0.349) is greater than the conventional significance level of 0.05, we fail to reject the null hypothesis (H1). This means that there is not enough evidence to conclude that there is a significant difference in employees’ motivation between the two types of VPAs.

In other words, the study did not find a statistically significant difference in motivation levels when using the UC software with a task-oriented VPA compared to a motivator VPA.

Satisfaction

Based on the results of the independent-samples t-test, there was no significant difference in employees' satisfaction when using the UC software with VPA between the task-oriented VPA and the motivator VPA. The t-value was 1.423, and the associated p-value was 0.162.

Since the p-value (0.162) is greater than the conventional significance level of 0.05, we fail to reject the null hypothesis (H2). This means that there is not enough evidence to conclude that there is a significant difference in employees' satisfaction between the two types of VPAs.

In other words, the study did not find a statistically significant difference in satisfaction levels when using the UC software with a task-oriented VPA compared to a motivator VPA.

Usability

Based on the results of the independent-samples t-test, there was no significant difference in usability in UC software with VPA between the task-oriented VPA and the motivator VPA. The t-value was 0.023, and the associated p-value was 0.981.

Since the p-value (0.981) is greater than the conventional significance level of 0.05, we fail to reject the null hypothesis (H3). This means that there is not enough evidence to conclude that there is a significant difference in usability between the two types of VPAs.

In other words, the study did not find a statistically significant difference in usability when using the UC software with a task-oriented VPA compared to a motivator VPA.

Nonparametric correlations

The Spearman rank-order correlation coefficient as a nonparametric measure of the strength and direction of association that exists between two variables measured on at least an ordinal scale was used in the analyses. The correlation was used to investigate whether there is a relationship between the VPAs’ usability and motivation, and VPAs’ usability and satisfaction. Spearman’s correlation was applied with the following assumptions:

both variables were measured on an ordinal scale (Likert scale 1–5)
the variables represent paired observations, for usability and motivation, usability and satisfaction, a single paired observation reflects the score on each variable for a single participant
there is a monotonic relationship between the two variables.

H4: There will be a significant relationship between VPA’s usability and employees’ motivation when using the UC software with both VPAs.

There was a positive correlation between usability and motivation, which was statistically significant (rs (42) = .354, p = .015). Hypothesis H4 is retained.

H5: There will be a significant relationship between employees’ motivation and satisfaction when using the UC software with both VPAs (Fig. 63 & 64).

There was a positive correlation between satisfaction and motivation, which was statistically significant (rs (42) = .469, p = .001).

Therefore, both H4 and H5 are retained. The results indicate that there is a significant positive relationship between VPA’s usability and employees’ motivation, as well as between employees’ motivation and satisfaction when using the UC software with both types of VPAs.

Attitudinal questions

The Likert scale questions regarding the impact of the VPA on motivation, satisfaction, communication, and collaboration were assessed using a 5-point scale, ranging from 1 (strongly disagree) to 5 (strongly agree). Participants were asked to rate their likelihood of experiencing certain outcomes by using the VPA. The questions and scale used were as follows:

1. How likely are you to increase your motivation by using this VPA?

2. How likely are you to increase your satisfaction by using this VPA?

3. How likely are you that by using this VPA your communication at work will be more effective.

4. How likely are you that your collaboration at work will be more effective by using this VPA.

5. How likely are you that by using this VPA your communication at work will be easier.

6. How likely are you that by using this VPA your collaboration at work will be easier.

Participants were asked to indicate their level of agreement or disagreement with each statement by selecting the appropriate response option on the Likert scale.

Based on the participants’ responses to the attitudinal questions, it appears that there is mixed perception regarding the impact of the task-oriented VPA in UC software on their motivation, satisfaction, communication, and collaboration. Here is a summary of the findings:

1. Motivation: Only 27% of participants think that using the VPA will likely increase their motivation, while a larger proportion (41%) does not see a likely impact on their motivation.

2. Satisfaction: 50% of participants believe that using the VPA will likely increase their satisfaction.

3. Communication Effectiveness: 41% of participants think that using the VPA will likely make their communication at work more effective.

4. Collaboration Effectiveness: Similar to communication, 41% of participants believe that using the VPA will likely improve their collaboration at work.

5. Ease of Communication: A majority of participants (59%) think that using the VPA in UC software will likely make their communication at work easier.

6. Ease of Collaboration: An even larger majority (64%) believes that using the VPA will likely make their collaboration at work easier.

These results suggest that while participants see potential benefits in terms of satisfaction, communication ease, and collaboration ease, they are less convinced about the impact on their motivation and the overall effectiveness of communication and collaboration. It may be valuable to further explore the reasons behind these perceptions and gather additional feedback to inform future improvements in VPA design and functionality.