My prediction: @StateBarCA will announce this week that the online California bar exam is cancelled. My guess is that this decision has already largely been made, and currently they are primarily talking about fallout and what’s next: 1/many

Whether or not the California authorities who regulate the profession are ready to admit it to themselves or not, a bar exam can not be successfully and fairly administered, from either a technology, privacy, or an artificial intelligence perspective. 2/n

Last Tuesday, July 7, three Justices of @CASupremeCourt, and representatives from @StateBarCA, its Board of Trustees, and its Committee of Bar Examiners held a public meeting regarding a potential second 2020 California bar exam. 3/n

I am grateful to @DiplomaPriv4All for allowing me to contribute to the Report they filed in advance of that meeting. In one section of the Report, we detail some of the troubling technology, privacy, and artificial intelligence issues that the online bar exam presents. 4/n

How does a remotely-proctored examination work? As you take the exam, your webcam records a video of your face, and the audio in your environment. This gets uploaded to the vendor’s servers for analysis. 5/n

This upload happens either in real-time (with a constant internet connection), or the video is stored on your PC and uploaded later (e.g.,with your exam files). 6/n

But it isn’t quite so easy as @StateBarCA saying “you won’t need a constant internet connection.” You need that, or a PC that can handle the added overhead of saving (and most likely compressing/encrypting) the video of you taking the test, while you take the test. 7/n

It sounds like this video will likely be used for facial recognition, so that you can be checked in to later bar exam sessions automatically with your face. So most likely the analyzed video is “unique biometric data” under #CCPA @ASmEdChau @caprivacyorg @eff @XavierBecerra 8/n

Generally, facial recognition systems have tended to work most accurately for white males. So these examinees may be less likely to be confronted with frustrating “technical issues” during the bar exam. 9/n

According to @StateBarCA’s original May 11, 2020 Workplan for administration of the online examination, the planned date for a decision on whether to proceed with attempting an online bar exam (“go/no-go”) was July 10 (although certainly this date could have been extended). 10/n

In my view it would be an understatement to say that the Workplan’s timeline for the development and deployment of this complex client-server application is compressed. 11/n

The rapid timeline, and/or the sheer scale of the examinee pool, has apparently necessitated certain compromises. 12/n

For example, @StateBarCA’s description of the planned heavy reliance on post-exam review of AI-flagged video events (rather than live human proctoring) made it sound as if CalBar may have selected the vendor’s proctoring service for “lower- to mid-stakes assessments…” 13/n

..rather than the live human proctor service that the vendor deems “essential for high-stakes programs.” 14/n

After all the video of you taking the test is uploaded to the vendor’s servers, what do they do with it? Well, for one, they use it to improve their AI product, and presumably you will be forced to “consent” to this in order to begin taking the test. @CITRISPolicyLab 15/n

It’s one thing to have to agree to the vendor’s click-through contract of adhesion when you have the choice to hand-write the test. Agreeing when using this service is the only path to licensure doesn’t feel very consensual. 16/n

Of course, humans are not going to watch all 65,000 hours of recordings of people’s faces as they take the bar exam. So events deemed suspicious by AI will be flagged by AI for human review. @CITRISPolicyLab 17/n

While @StateBarCA’s vendor advises that for high-stakes exams it’s preferable that potential cheaters are interrupted in real time during the exam, it isn’t clear whether @StateBarCA has contracted for the vendor to supply sufficient human proctors for that. @CITRISPolicyLab 18/n

Given the extent of coverage of bias inherent in some AI systems due to homogenous input/training data, it would be valuable to know what due diligence has been performed to determine how bias has been avoided by CalBar’s vendor. @CITRISPolicyLab @BNonnecken @ACLU_NorCal 19/n

Under Calif. R. of Court 9.6, the Committee of Bar Examiners is responsible for determining how California’s bar examinations should be conducted, subject to approval by the @CASupremeCourt. 20/n

In the past few days, a couple (of the 17) Committee members have voiced or renewed support for #DiplomaPrivilege. Why now? Conceivably because they have, or are about to, recommend to the @CASupremeCourt that the online CBX should be cancelled. 21/n

This is pure speculation -- Kremlinology. But arguably, under the circumstances these two weeks should be a period of radio silence on policy. Because otherwise people may engage in Kremlinology :^\\ 22/n

Of course, without an online bar examination or alternative route to admission such as #DiplomaPrivilege, and assuming that the current health emergency continues to preclude an in-person bar exam, this means no more new California lawyers in 2020, or even well into 2021. 23/n

But that’s @CASupremeCourt's call, not the Committee’s. It’s nobody’s fault that an online bar exam just isn’t possible on this timeline/scale. (Really, it’s not @StateBarCA’s fault -- vendors are scrambling to meet demand that has only existed for a few months.) 24/n

If you’re an applicant upset there’s not going to be an online bar exam, in my view there was never going to be -- not this year, anyway. Wouldn’t you rather find that out this week, rather than after you spend even more time rearranging your life around an imaginary event? 25/n

I could be wrong -- many states may still try to go ahead with an online exam. Except perhaps for very small examinee pools (Indiana has approx. 500), I predict that states attempting mass online exams will come to regret it, overall. 26/n

If nothing else, people should be mentally prepared for a possibility that many don’t seem to seriously contemplate -- that there will be no further possibility of becoming a lawyer in California in 2020. I certainly hope I’m wrong, believe me. 27/n

What if you were planning on being fine taking the online exam -- your technology is all lined up, with your Fiber-to-the-Home broadband, automatic failover to the hot site in the shipping container with a diesel generator out back, and your screaming new gaming laptop? 28/n

I’ve taken a couple of remotely-proctored professional certification tests recently, and my tech is all lined up too. Personally, I still don’t trust the client software, or the server side, to not arbitrarily crash me out of the test. 29/n

Even in an imaginary world in which I could guarantee that the technology would all work fine for me personally, I like to believe that I’d prefer no exam to one that adds further discrimination against examinees that don’t have access to those same resources that I might. 30/n

I also like to think that in that imaginary world, I’d also prefer that there be no online California bar exam, instead of one that further normalizes the role of Big A.I. Testing in our society. @ACLU_NorCal 31/n