Background
Recent research shows that the use of body-worn cameras (BWC) alone does not significantly improve police performance and accountability. Specifically, there remains substantial uncertainty about whether BWC can alone reduce officer use of force. Cynthia Lum, et al. have shown a lack of conclusive evidence that police behaviors change solely from the use of body worn cameras and instead suggest that the use of BWC for coaching, training, and accountability could successfully deliver on these promises.¹
Plain Language Summary
Automated BWC analytics provides a method for departments to review 100% of their BWC videos to surface positive interactions that reinforce professionalism training and correct risky behavior as it occurs, fulfilling the promise of BWC. In this study, we demonstrate that the use of BWC analytics combined with professionalism training leads to substantial reduction in use of force occurrences (across de minimis through lethal use of force) as well as an increase in positive behavioral changes in both officers and the community members they serve.
This study presents an analysis of BWC data from the police department of Alameda, California comparing the first half (H1) of 2021 vs first half of 2022. In H2 of 2021, Alameda PD Chief Nishant Joshi implemented several training programs centered around strategic communication and began use of Truleo’s automated BWC analysis software. Prior to Truleo, Alameda PD was leveraging Sergeants to conduct random audits that resulted in reviewing less than 1% of videos. After training and automated BWC analytics were implemented, we observed the following changes in the Alameda PD data:
36%
decrease in use of force
30%
decrease in
unprofessional officer language
17%
increase in
officer explanations
12%
decrease in
civilian non-compliance
99%
of interactions were at a standard or high professionalism level
Author Conclusion
These results provide strong evidence that the implementation of strategic communication training and automated BWC analytics can reduce the number of use of force occurrences, improve officer professionalism and, as a result, improve civilian compliance. Moreover, we find that when officers use unprofessional language in an effort to de-escalate, this language does not reduce civilian non-compliance.
I commend our leadership team, from command staff to front-line supervisors to the officers on the front line. They all bought into the mission and leveraged BWC analytics as a coaching tool to help us create better outcomes with our community. The data helped us celebrate the 99% of interactions that were professional while learning how to improve from the other 1%.
CHIEF NISHANT JOSHI
ALAMEDA POLICE DEPARTMENT
Implementation of Training
and BWC Analytics
As many cities struggle with increasing tensions between police and the communities they serve, research has suggested that more professional behavior on the part of police officers can lead to more effective policing. In particular, research suggests that officers using polite language and providing explanation of actions before they occur can lead to more positive interactions with civilians.²,³ Increases in positive interactions between police and community members have also been shown to lead to a general improvement in civilian compliance with police.⁴
From the start of Chief Joshi’s tenure at Alameda PD in June of 2021, several policy changes were implemented:
01.
Leadership across the department (command staff through front-line supervisors) prioritized body-worn camera activations for every public encounter and codified policy consequences for high failure of activation.
02.
As of December 2022, 67% of Alameda patrol officers are new, a net change of 21 patrol officers in addition to 8 of 14 promoted Sergeants and 4 of 6 promoted Lieutenants. This large change provided an opportunity for new training policies to have a significant influence on new recruits.
03.
Leadership implemented an over 10x increase in required training hours, from 30 hours every 2 years to 330 hours every 2 years. A core emphasis of the additional training was de-escalation and communication training.
04.
In January of 2022, Alameda implemented Truleo BWC analytics. Highly professional interactions were surfaced by Truleo’s software, and command staff sent department-wide communications congratulating good officer behavior as a way to reinforce strategic communication training.
05.
Following the implementation of BWC analytics, Sergeants at Alameda PD began using a precision-guided audit workflow (in contrast to random audits). Truleo’s BWC analytics would automatically surface interactions with potential risky behavior, allowing the reviewer to go to the precise moment of officer unprofessional language and determine if it was risk or not, enabling the review of key moments across hundreds of videos in the time a single video would take to review.
Each value is normalized to audio hours of interaction, where an interaction audio hour contains speech from either an officer or a civilian. An instance of a metric is a single statement by an officer or civilian that indicates that label.
Impact of Policy Measured by BWC Analytics
Chief Joshi began his tenure at Alameda Police Department in June of 2021. To make a fair comparison, this study compares data from January 1st - June 30th of 2021 (H1 2021) to the same time period comprising H1 2022. In this way, any seasonal changes in officer or civilian behavior, as well as significant world events and their effect, are disregarded.
This study conducted BWC analysis using machine-automated speech recognition, speaker diarization, officer voice and language identification, and natural language processing. All machine-learned models in this process used derivations of transformer-based deep-learning models, which understand language based on local context.⁵
Table 1 shows a summary of key metrics measured from BWC analysis during the two 6-month time periods and the overall percentage change between the time periods. Below we summarize the key results.
Impact of Policy Measured by BWC Analytics
Compared to H1 2021, H1 2022 showed a 55% increase in raw BWC footage duration and a 104% increase in BWC interaction duration. BWC footage duration increase suggests that officers are turning on their cameras more often than previously. BWC interaction duration increase indicates that the times where officers are turning on their cameras are overwhelmingly ones that have officer and civilian speech rather than just silence or background noise.
The department policy emphasis on maintaining high numbers of camera activations likely resulted in the observed increase in BWC interactions.
Comparative Improvement in Professionalism
Between H1 2021 and H1 2022, this study observed a 17% increase in the number of words of explanation officers were delivering per hour of BWC footage on average. In concert with this increase in explanation, we observed a 12% reduction in civilian non-compliance between H1 2021 and H1 2022.
We propose a definition for a “standard professional” interaction as one with no unprofessional officer language, and a “highly professional” interaction as having three conditions: (1) no unprofessional language from the officer, (2) 25 words or more of explanation used before a use of force event, and (3) using at least one instance of polite language. Using this definition, the resulting percentage of highly professional interactions increased from 10% in H1 2021 to 12% in H1 2022, a 20% relative increase in the number of highly professional interactions. Over both periods, the number of professional interactions (standard professional + highly professional) was 98% and 99%, respectively, indicating a 50% reduction in unprofessional interactions.
This study also observed a 36% decrease in language indicative use of force from H1 2021 to H1 2022, which includes all types of use of force. Alameda PD reports all use of force and conducts follow-up investigations in incidents with injuries or complaints. In these follow-up investigations, the agency observed a 70% reduction over the same time period, corroborating the trend observed via BWC analytics.
The measure of use of force through language can come from either the civilian or the officer as an indication a use of force is occurring.
In parallel, this study observed a 30% decrease in officer use of unprofessional language between H1 2021 and H1 2022, despite a 27% increase in civilian usage of unprofessional language comparing the same time periods. Here, unprofessional language is defined as the use of profanity, insults, and threats.
Overall, the increase in officer explanation and reduction in officer unprofessional language suggests a concerted effort from officers to improve the professionalism of civilian interactions with strategic language, rewarded with a decrease in civilian noncompliance and decreased need for use of force from officers.
Unprofessional language fails to de-escalate scenarios
In addition to the comparative broad results of increased professionalism in the Alameda Police Department, this study also analyzed interactions across the same time period to understand officer use of unprofessional language.
Officers tend to deploy unprofessional language as an attempt to de-escalate a noncompliant civilian. However, in analyzing these interactions, we find that the use of unprofessional language fails to decrease civilian noncompliance.
The percentage of civilian language in recorded BWC containing instances of noncompliant language (the noncompliance rate) across all interactions was 2.5%.
As Figure 1 shows, the rate of civilian noncompliance increases after even one occurrence of unprofessional officer language.
While a 3x increase of noncompliance occurs after even 2 occurrences of unprofessional language, 4 or more occurrences increase non-compliance by 8x to 20x to a maximum of 35% non-compliance. This means for interactions with 6 or more instances of unprofessional language from an officer, 1 out of 3 civilians is likely to be non-compliant.
We note that it cannot be concluded from this data that the officer’s use of unprofessional language was the cause of further noncompliance, but these data clearly demonstrates that unprofessional language does not assist an officer in obtaining better civilian compliance.
Methods
Data Collection and Analysis
The data for this study were collected via an API provided by the department’s evidence management system. Videos delivered via the API were fed through an audio extraction pipeline such that only the audio information of the video was retained in temporary memory for analysis. Audio was fed through machine-learned speech recognition models to obtain words and timestamps of valid speech in the audio file. Video files that contained only noise or silence were discarded during the audio analysis process.
DATA COLLECTION AND ANALYSIS
Many accuracy metrics are available for BWC analysis models, including precision, recall, f-beta, and conventional “accuracy”. For this study, precision was chosen as the metric of accuracy to be optimized across all models.
By nature, BWC videos will not capture all events as the quantity of video depends on an officer physically turning on their BWC. This puts an undetermined limit on recall, which is the ability of a model to correctly identify all events or labels that may have occurred in all known interactions. Late activation and early termination of BWC video by the officer can exacerbate these issues. Additionally, even in a recorded interaction, it is possible that an event occurs without verbal cues, such as a use of force.
Knowing that all interactions may not be captured, a comparative analysis can still be conducted assuming the characteristics listed above do not significantly change. In these comparative analyses, precision (the number of true positives discounting false positives, ignoring false negatives) is the best metric as relative values can be compared with confidence.
OFFICER IDENTIFICATION AND ACCURACY
Subsequent analysis on the audio was conducted to diarize the audio into anonymous speakers, and a language-based model was used to identify which of the anonymous speakers was likely to be the officer wearing the camera. In testing, this model performed with an 85% level of precision. The remaining speakers were aggregated into civilian audio. The same model was used across all analyses, enabling confident comparisons of relative differences acknowledging that absolute numbers may be affected by precision errors.
NATURAL LANGUAGE PROCESSING EVENTS, LABELS, AND ACCURACY
After officer / civilian identification, the resulting text was analyzed via natural language processing (NLP) models that identified certain events and labels.
NLP models were trained from data sets where humans tagged segments of transcribed text with relevant events and labels, with a target inter-annotator agreement level exceeding 90%.
Events were output via an intent classification model taking a segment of text from 1 to 50 words from a single utterance of a single speaker and outputting a single label. Labels were output via either an intent classification model or a named entity recognition model that tags one or more words with a specific entity tag.
Events were defined as language that indicated an event occurred in a BWC interaction, whether from the civilian or the officer. The two events relevant to this study were “use of force” and “non-compliance”.
Language for both of these events could occur via either an officer or civilian. Use of force language was collected to cover the entire range of use of force, from de minimis force through lethal force. Some examples of language indicating use of force include “taser deployed” on behalf of the officer or “don’t tase me” on behalf of the civilian. Similarly, “you’re still refusing to cooperate” and “I won’t comply” represent the language of “noncompliance” from the angle of the officer and civilian, respectively.
Labels were classified as professional or rude language. Professional language was dissected into the use of “polite” language and the use of explanation. Polite language includes the use of formality, gratitude, and politeness, such as “Mr. / Sir / Ma’am”, “Thank You”, and “Please”, respectively.
Explanation is defined as instances of officers explaining to a civilian why they are taking a certain action (such as a frisk, detention, or arrest) using more than 25 words.
Rude language is defined as profanity (whether directed or not - “f*** you” or “s***”, respectively), insults (racial slurs), or threats (“I’m going to kill you”).
In testing, the NLP precision for this study was over 90% for “use of force”, “noncompliance”, “threat”, “profanity”, “politeness”, and “explanation”.
Citations
1) Lum, C. et al., “Body-worn cameras’ effects on police officers and citizen behavior: A systematic review”, Campbell Systematic Reviews 2020 (https://onlinelibrary.wiley.com/doi/10.1002/cl2.1112)
3) Wood, G. et al., "Procedural justice training reduces police use of force and complaints against officers", PNAS 2020 (https://doi.org/10.1073/pnas.1920671117)
2) Tyler, T. "Legitimacy and Procedural Justice: A New Element of Police Leadership", 2014 (https://www.ojp.gov/ncjrs/virtual-library/abstracts/legitimacy-and-procedural-justice-new-element-police-leadership)
4) Peyton, K. et al., “A field experiment on community policing and police legitimacy”, PNAS 2019 (https://doi.org/10.1073/pnas.1910157116)
Author Affiliations
Truleo Inc.
1 E Erie Street, Suite 525-2246
Chicago, IL 60611
SHARE CASE STUDY
About Truleo
Truleo analyzes police body camera videos using artificial intelligence to help promote police professionalism. Truleo worked with FBI National Academy alumni to build the models that deconstruct language used during events to help agencies promote best practices, train new officers, and mitigate risk.
Learn more about Truleo
