UTA Improving Access to Justice through the strategic analysis of litigation events

Using our machine-learning tool, we conducted a beta test to engage in large-scale analysis of federal court records, examining thousands of pages of data in an attempt to answer public interest attorneys’ questions about access to justice issues and provide guidance on where federal courts in Texas fail to operate justly. The overall goal was to assist public interest attorneys as they advocate for systemic change in the court system. After downloading the needed court records, our team engaged in the following technological improvements to ready our tool for a public beta test: · Developed an ontology of 23 additional litigation events relevant to determining the outcome of a case. · Trained deep learning classifiers for each of these events. · Created training data and developed a standardized set of sampling techniques to reduce annotation labor and improve out-of-domain generalization. · Used unsupervised, contrastive learning techniques to develop a docket entry similarity model, that could be used to identify tricky cases within and outside of the training set. · Conducted several rounds of evaluation and refinement to identify and correct points of error, iterating until all models achieved 0.985+ F-scores. · Operationalized sampling strategies in an easy-to-use web application. · Created a Python module with inference pipelines, allowing users to make predictions with models in two lines of code. · Computed model predictions across the entire SCALES corpus of 11M+ docket entries. Summarized results by representing cases in terms of ‘pathways’ — sequences of relevant events at various levels of granularity to provide a concise view of how the lifecycle of cases in the federal court system. · Used pathway data to further refine underlying models, by identifying unexpected sequences of events and reviewing their predictions. In addition, we hosted several demonstrations, workshops and panels, where we made the tool publicly available for the first time, allowing users to begin to query court records independently. We continue to refine the tool based on feedback from community partners and other users.