Our research is largely inspired by the ubiquitous nature of today's technologies, and how smart sensing enabled by these technologies can be used to understand how people interact with the technology itself, interact with each other, and behave in different environments. To do so, we generally rely on applied artificial intelligence, statistics, and human-centered computing techniques to answer research questions which focus on

• usable and inclusive cybersecurity systems for personally-owned computing devices, particularly related to identity and access management
• interdisciplinary applications of human behavior analysis through smart sensing

Here are some themes and techniques that we currently work on:

Mobile Biometrics

Commercial mobile devices currently employ point-of-entry (PoE) authentication methods such as face recognition and personal identification numbers. Several studies show that knowledge-based authenticators are ineffective due to increased memory load, poorly chosen alphanumeric combinations, and repetitive use of passcodes across many applications. PoE and knowledge-based authenticators are both ineffective throughout sessions of use; once the user is authenticated, the device and its content remain available to the current user. In this research, we explored various data based on usage of a mobile device as behavioral biometric modalities, including patterns in application use, networking activity (i.e., Bluetooth and Wi-Fi sightings), and communication through calling and text messaging, which can continuously and passively authenticate a device’s owner.

Significant findings and methods developed throughout this research include:

• We demonstrated the use of association rules as a feature representation for application, Bluetooth, and Wi-Fi data which outperformed previously employed frequency-based features. Our findings suggest that patterns in application and Bluetooth traffic are more distinct than those in Wi-Fi patterns with up to 91% accuracy. Performance obtained in this study were consistent with implementations of keystroke dynamics and touch gestures for biometric authentication, a significant indication that passive modalities are as viable as their physical counterparts.
• We provided empirically-supported evidence of gender-based usage patterns, including higher activity levels of female compared to male users in application use and the differentiation between the two groups based on transitions between sports, navigation, and system-related apps and number of revisitations to apps within sessions.
• We developed temporally-derived replay attack simulations to assess system performance under threatening conditions in which the legitimate user is easily observed or their usage data is remotely accessed. The intercepted data is then intelligently combined with some level of noise to avoid the replay of an exact copy of legitimate data by an attacker. The attacker’s data thus maintains a level of similarity with the legitimate data. Our experiments show that our attack models can lead to false positive rates ranging from 30 to 50%, while in some cases, zero-effort attacks lead to much lower, and thus misleading, error rates.
• We introduced an easily interpretable profiling technique as a feature representation for user-device interaction data that contextualizes mobile device usage patterns. Our experiments show that these profiles allow better separation of legitimate and impostor behavior. Error rates associated with behavior profiles either improved or were not affected by the amount of data provided for authentication, while those associated with frequency-based features, which are seen in several research studies, worsened.
• We developed a novel ranking function which quantifies the informativeness of a soft biometric class for search space reduction. The function relies on the rank-1 recognition accuracy for the class, a score quantifying the permanence of the class, and a cost derived from the ratio of misclassifications to the number of subjects in the soft biometric class.

---

Age-Aware User Authentication

Current software for user authentication relies on the user to directly initiate some interaction (i.e., active authentication). However, active authentication systems are not accessible to individuals across all age groups. Continuous authentication schemes transparently observe a user's natural multimodal behaviors to leverage all possible signals as input for authentication, and hence do not require explicit authentication interactions to be initiated by the user, and are thus a promising framework for authentication by individuals of different age groups. This project's novelties are 1) to advance understanding of how individuals of different age groups use and perceive existing authentication methods, especially concerning users' mental models and acceptance of monitoring for the purposes of continuous authentication, and 2) to collect and analyze a variety of user signals in multiple behavioral and physiological modalities for age-aware continuous authentication on personal computing devices. This research also informs the design of continuous authentication interactions in other contexts such as public spaces and other smart environments, in which continuous authentication might be useful. The research includes three phases. (1) Elicit the mental models multi-generational users have of what it means to authenticate to a system, if and when they expect the system to re-authenticate them to confirm their identity as they continue to interact, and if and how they expect to receive feedback of authentication attempts. (2) Produce a novel dataset of behavioral and physiological data, such as touch gestures, keystroke dynamics, heart-rate variability, and skin temperature, through a series of data collection sessions wherein individuals of different age groups will be recruited to complete a diverse set of tasks. (3) Develop fundamental knowledge of age-aware continuous authentication through the analysis of these data using state-of-the-art machine and deep learning techniques.

This project is a collaboration between the CIBeR Lab, the Affective Vision Lab, the Intelligent Natural Interaction Technology Lab, and the Ruiz Human-Computer Interaction Lab.

Here's a research talk that Dr. Neal gave at the Pacific Northwest National Laboratory focused on this project.

The material produced in this project is based upon work supported by the National Science Foundation under Grant No. 2039373 and 2039379. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This study has been approved by the USF IRB board #STUDY002291.

This project received Research Experience for Teachers and Undergraduates supplements through the National Science Foundation to engage with undergraduate students on the development of high school cybersecurity teaching materials. Read the results here.

---

Human Behavior Tracking and Mental Health

This project focuses on the Up To Me program, a three lesson, disclosure-based stigma reduction program meant to reduce barriers to community living and participation for college students with psychiatric disabilities. Anticipated outcomes of this project for college students include improvements in 1) self-stigma (self-esteem and self-efficacy), 2) empowerment and self-determination, 3) appraisals of stigma as a stressor, 4) student engagement on campus, 5) sense of campus belonging, and 6) care seeking/service engagement for mental illness. The CIBeR Lab's contributions lies in the investigation of the use of sensor data collected from a smartphone as a proxy of behavior to substantiate self-report data on outcomes.

This project is a collaboration between the Stigma Action Research Lab and the CIBeR Lab.

This award (90IFRE0056) is supported by the Administration for Community Living (ACL), U.S. Department of Health and Human Services (HHS) as part of a financial assistance award totaling $600,000 with 100 percent funding by ACL/HHS. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by ACL/HHS, or the U.S. Government.

---

Transportation Behavior Analysis via Smart Sensing

Multimodal transportation, such as transit, bike, walk, ride-hailing (e.g., Uber, Lyft), car-share, and bike-share, are vital to supporting livable communities. However, activity data of travelers using these modes have been difficult to acquire. This research leverages ground truth data to develop intelligent models to infer missing and incomplete travel behavior characteristics from the OneBusAway data to provide trip origin, destination, and mode. The research team will deploy these models to provide an enhanced picture of the travel behavior of OneBusAway users in Mayaguez, Puerto Rico. The team will use the database generated by the intelligent model and additional transportation system information to conduct demand analysis and identify demand patterns and their impact on traffic congestion. The resulting model and tools will be immediately applicable to all nine cities that have deployed OneBusAway to better understand the relationship between multimodal travel behavior and traffic congestion. Using crowdsourcing, big data science, and machine learning, these tools will help transportation agencies deploy new, automated strategies to improve congested multimodal systems in the immediate future.

This project is a collaboration between the Center for Urban Transportation at USF, the University of Puerto Rico at Mayaguez, and the CIBeR Lab.

This project, 3-7: Transforming multimodal travel behavior data from an open-source platform to support traffic congestion reduction strategies, is supported by the National Institute for Congestion Research.

---

We also conduct research in natural language processing as a part of the Natural Language Processing Group at USF.

Open positions

Currently, we do not have funding for additional PhD students and can only welcome you if you have a fellowship.

Students interested in joining CIBeR are encouraged to review the lab's manual to determine if our lab is a good fit for you. The content below provides more information about joining our group.​

Applications for PhD positions

Ph.D. students with experience in one or more of the following research areas are encouraged to send their application materials (listed below) to Dr. Neal via e-mail.

• Machine and deep learning / pattern recognition systems
• Statistical analysis
• Image processing / computer vision
• Qualitative data analysis methods
• Natural language processing
• Proficiency in Python, TensorFlow, Keras, Sci-kit Learn
• Cloud storage
• Data mining

All graduate students must be able and willing to:

• Support undergraduate students / serve as mentors,
• Work toward effective communication of their research (e.g., presentation and writing skills),
• and work peacefully and productively in a team environment.

Interested students should submit the following materials via e-mail for consideration.

1. A brief summary of two projects (can be coursework or something you have completed on a job or your own) that you are most proud of, and what your contributions were to the project.
2. Your views on the on-going research in the CIBeR lab that appeals to you the most. This requires that you familiarize yourself with our research projects. I'd like your input on our current research strategies, and how you feel you might be able to expand or contribute to these efforts.
3. Brief write-up of how you feel technology and computer science can be used to improve problems facing the world. Please provide specific examples.
4. List of courses taken with final grades.
5. Degree you are currently pursuing and expected start date at USF.
6. Updated CV and publications, presentation slides, etc, if available.
7. Link to personal website.

Undergraduate Researchers

We are always looking for enthusiastic undergraduate students interested in gaining hands-on experience in research. Ideal students will have some experience in our research areas and will be able to clearly articulate how they want to contribute to the lab. Ideal students will also be able to dedicate 3-5 hours to working with our research team per week, and will meet regularly during lab meetings. All undergraduate students are brought into the lab as volunteers, and are encouraged to take full advantage of the wide spectrum of learning opportunities. Importantly, all undergraduates working with the lab should strongly considering registering for Supervised Research (CIS 4915) credits.

M.S. students seeking a thesis advisor are encouraged to reach out to Dr. Neal no later than the middle of their first semester in the M.S. program.

Our laboratory frequently conducts research studies that require input from participants.

We are currently seeking participants for the following studies:

• Study 005331: Smart Door Authentication System (Download Recruitment Flyer)
  Lead Researcher: Mohamed Ebraheem
• Study 006481: I-CLAIM: Focus Groups: Assessing User Authentication Experiences with Historically Excluded Groups (Download Recruitment Flyer)
  Lead Researcher: Hoorad Abootalebi

Refereed Journal Articles

1. Negro A, Montagna F, Teng MN, Neal T, Thomas S and King S (2023) "Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs." Front. Res. Metr. Anal. 8:1204801. doi: 10.3389/frma.2023.1204801

2. Chaudhary, M., Kosyluk, K., Thomas, S., and Neal, T. "On the use of aspect-based sentiment analysis of Twitter data to explore the experiences of African Americans during COVID-19." Sci Rep 13, 10694 (2023). https://doi.org/10.1038/s41598-023-37592-1.

3. Kristin A. Kosyluk, Jennifer T. Tran, Sayde King, Katie Torres and Tempestt Neal (2023) "Preliminary effectiveness study of the Cope Notes digital mental Health program", Journal of Mental Health, DOI: 10.1080/09638237.2023.2182424.

4. Sayde Leya King, Jana Lebert, Lacey Anne Karpisek, Amelia Phillips, Tempestt Neal, and Kristin Kosyluk. "Characterizing user experiences with an sms text messaging–based mhealth intervention: Mixed methods study." JMIR Form Res, volume 6, page e35699, May 2022. https://doi.org/10.2196/35699.

5. Tempestt Neal and Damon L. Woodard. "You are not acting like yourself: A study on soft biometric classification, person identification, and mobile device use." IEEE Transactions on Biometrics, Behavior, and Identity Science, volume 1, pages 109–122, 2019.

Refereed Conference and Workshop Articles

1. Saandeep Lakshminarayan, Meghna Chaudhary, Tempestt Neal, and Shaun Canavan. Multimodal Context-based Continuous Authentication. 2023 IEEE International Joint Conference on Biometrics (IJCB). Accepted (Paper ID: IJCB2023-179).

2. King, S.L., Johnson, N., Kosyluk, K., Neal, T. (2023). Therapist Perceptions of Automated Deception Detection in Mental Health Applications. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2023. Lecture Notes in Computer Science(), vol 14050. Springer, Cham. https://doi.org/10.1007/978-3-031-35891-3_6.

3. Lozano, W., King, S., Neal, T. (2023). Observations of Caregivers of Persons with Dementia: A Qualitative Study to Assess the Feasibility of Behavior Recognition Using AI for Supporting At-Home Care. In: Gao, Q., Zhou, J. (eds) Human Aspects of IT for the Aged Population. HCII 2023. Lecture Notes in Computer Science, vol 14043. Springer, Cham. https://doi.org/10.1007/978-3-031-34917-1_23.

4. N. Loecher, S. King, J. Cabo, T. Neal and K. Kosyluk, "Assessing the Efficacy of a Self-Stigma Reduction Mental Health Program with Mobile Biometrics: Work-in-Progress," 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), Waikoloa Beach, HI, USA, 2023, pp. 1-6, doi: 10.1109/FG57933.2023.10042655.

5. Tempestt Neal, Lisa Anthony, Shaun Canavan, Jaime Ruiz, Saandeep Aathreya, Meghna Chaudhary, Yu-Peng Chen, Heting Wang, Rodrigo Calvo, Liza Jivnani, and Nicolas Ng Wai. Toward understanding children’s use and understanding of user authentication systems: Work-in-progress. In USENIX Symposium on Usable Privacy and Security (SOUPS), Boston, MA, USA, August 2022.

6. Parush Gera and Tempestt Neal. A comparative analysis of stance detection approaches and datasets. In Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems, pages 58–69, Online, November 2022. Association for Computational Linguistics. https://aclanthology.org/2022.eval4nlp-1.7.pdf.

7. Mohamed Ebraheem, Sayde King, and Tempestt Neal. Lip movement as a wifi-enabled behavioral biometric: A pilot study. In Constantine Stephanidis, Margherita Antona, and Stavroula Ntoa, editors, HCI International 2022 Posters, pages 473–480, Cham, 2022. Springer International Publishing.

8. Khadija Zanna, Tempestt Neal, and Shaun Canavan. Clustering of physiological signals by emotional state, race, and sex. In Companion Publication of the 2021 International Conference on Multimodal Interaction, ICMI '21 Companion, page 312–316, New York, NY, USA, 2021. Association for Computing Machinery.

9. Matthew Sumpter and Tempestt Neal. User perceptions of article credibility warnings: Towards understanding the influence of journalists and AI agents. In MEDIATE 2021 in conjunction with the 15th International AAAI Conference on Web and Social Media (ICWSM), 2021.

10. SK Rahatul Jannat, Diego Fabiano, Shaun Canavan, and Tempestt Neal. Subject identification across large expression variations using 3D facial landmarks. In Alberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, and Roberto Vezzani (Eds.), Pattern Recognition. ICPR International Workshops and Challenges, pages 5–13, Cham, 2021. Springer International Publishing.

11. Tempestt Neal and Ashokkumar Patel. A brief literature review and survey of adult perceptions on biometric recognition for infants and toddlers. In 2020 IEEE International Joint Conference on Biometrics (IJCB), pages 1–10, 2020.

12. Tempestt Neal and Shaun Canavan. Mood versus identity: Studying the influence of affective states on mobile biometrics. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pages 562–566, 2020.

13. Sayde King, Mohamed Ebraheem, Khadija Zanna, and Tempestt Neal. Learning a privacy-preserving global feature set for mood classification using smartphone activity and sensor data. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pages 582–586, 2020.

14. Parush Gera, Nadia Thomas, and Tempestt Neal. Hesitation while posting: A cross-sectional survey of sensitive topics and opinion sharing on social media. In International Conference on Social Media and Society (SMSociety'20), page 134–140, New York, NY, USA, 2020. Association for Computing Machinery.

15. B. M. S. Bahar Talukder, Vineetha Menon, Biswajit Ray, Tempestt Neal, and Md Tauhidur Rahman. Towards the avoidance of counterfeit memory: Identifying the DRAM origin. In 2020 IEEE International Symposium on Hardware Oriented Security and Trust (HOST), pages 111–121, 2020.

16. Tempestt Neal and Damon Woodard. Mobile biometrics, replay attacks, and behavior profiling: An empirical analysis of impostor detection. In 2019 International Conference on Biometrics (ICB), pages 1–8, 2019.

17. Tempestt Neal, Md Asaduzzaman Noor, Parush Gera, Khadija Zanna, and Gurpreet Kaptan. Authenticating phone users using a gait-based histogram approach on mobile app sessions. In 2019 International Conference on Biometrics (ICB), pages 1–7, 2019.