This gallery contains 1 photo.
8 Reasons You Should Assign CALI Lessons to Your Students If you teach in a US law school, you should be looking at CALI Lessons and assigning them to your students. -1- CALI Lessons are another way to learn the … Continue reading
We’ve added a Question Bank feature to CALI QuizWright®, our formative assessment system. The new feature uses multiple choice questions selected from CALI Contracts Lessons. The 150+ contracts questions are available to everyone who uses QuizWright. Over the course of the fall semester we plan on adding more questions, both in contracts and in other areas of the first year curriculum including torts and property.
With one click you can add questions from the public Question Bank to your personal question bank. You are free to edit your copy of a question, customizing it for your class. Questions can be added to as many quizzes as you’d like.
QuizWright Question Bank
To get started faculty, law librarians, and staff at CALI member schools can log in to the CALI website and select QuizWright from the CALI Dashboard drop down menu. Once you’re in QuizWright click on Question Bank in the left hand menu. Clicking on the question in the list will show you details about the question including the answer choices, what CALI Lesson it came from, and who wrote the original question.
To add a question to your personal question bank click on the copy icon to the left of the question. The icon will change to a check mark indicating that you’ve successfully copied the question. To edit the copied question click on the My Questions link in the left hand menu. To use the copied questions in a quiz select Add a Quiz on the left and select the copied questions to include in a new quiz.
For more information please visit the about QuizWright page.
If you have any questions about QuizWright or the Question Bank feature just drop me an email: emasters AT cali.org
I just ran across another legal tech startup that is going to automate a common legal problem space and sell access to their web-based questionnaire for $29.95 or some such. It’s potentially a huge market because millions of people must deal with this legal situation every year. There are hundreds of possible startups like this – each centered around a single vertical that can be automated with a little javascript and a some marketing.
TurboTax was the first way back in 1987 (that’s when I first bought it) when it was a Lotus 1-2-3 spreadsheet with a ton of macros made by a company called Chipsoft. Now the 1040 and accompanying forms were and are hard to automate, but the market is huge (100 million potential uers) every year. There is a lot of law to keep up to date and it changes every year. Intuit bought Chipsoft for $223 million back in 1993.
Here’s my prediction. Every single “simple” legal problem that is process definable or form-oriented will go this way, but for much much less money. My thinking when we started working on A2J Author over 12 years ago is that courts and legal aid should do this for themselves and law students could help them by taking courses that include an experiential component. This would give law students the ability to be smart users, builders, and purchasers in the new normal of the automated process marketplace.
Even complicated legal matters can be broken down into steps – some that can be automated and some that shouldn’t. This could either make law practice more efficient or it could result in smaller, chunkier work for lawyers doing unbundled, limited license work.
This won’t put lawyers out of business either. When I started my first programming job in 1983 coding COBOL on an IBM mainframe, I read about a new code generator in ComputerWorld that was going to replace programmers. Instead, every new advance in coding has just meant that programmers had to change their development environment and upgrade their skills. This too shall happen to lawyers. It seems like it’s happening rapidly, but it’s actually going to take some time. Law moves slowly. Very slowly.
If this becomes a “race to the bottom”, then cutthroat competition will lower prices and maybe quality. This will make it very hard for companies to find sustainable income. There 
will be winners and losers and this might not be all good for the quality of legal service delivery. Even so, the market is not very good at measuring the quality of legal service delivery. Why is that?
Court forms are a kind of domain-specific language that courts and lawyers use to communicate with each other about legal matters. Courts have a monopoly on how the forms are formatted, but not on the guidance that is given in filling them out.
That is where lawyers add value and can differentiate themselves. Maybe automation and explanation is proof that the lawyer you are thinking of hiring actually understands what you are hiring her for.
John Mayer
Executive Director, CALI
@johnpmayer
jmayer@cali.org
Executive Director at Self-Represented Litigation Network
The rise of the self-represented litigant has disrupted the civil justice system. Courts no longer rely on lawyers to manage the litigants, but the due process remains so courts have had to step-up and create user-friendly systems for lay people. By providing comprehensive, 24/7 self-help services such as forms, instructions, tailored procedural guidance, and triaged case flow management; courts can create transparent and navigable systems. However, the bespoke approach contemplated in an adversarial process is lost without lawyers. Lawyers are still very much needed, however, their new role is only beginning to be understood. It is one that has paradoxically narrowed in focus yet, because of technology, expanded in delivery opportunities. Legal education has an opportunity to equip new lawyers with the legal and practical skills to be successful in today’s legal market that demands 24/7 services accessible by cellphone from anywhere in the world while engaging more autonomous clients who seek refined and targeted legal advice, strategy and big-picture analysis. This talk will explore the many opportunities that are presenting in this re-aligning market, and consider the negative and positive impacts, particularly with respect to technology, on access to justice.
With a deep background in designing and implementing access to justice initiatives for legal aid, the courts and private practice, Katherine’s philosophy throughout her career has been to build common sense, consumer oriented solutions by learning, innovating and sharing. After clerking for a trial court judge, Katherine worked at Alaska Legal Services Corporation, initially handling DV matters and later as the Aging Grant Coordinator. In 2001, Katherine joined the Alaska Court System to develop the statewide Family Law Self-Help Center, which resulted in the nation’s first virtual self-help center. Operating solely through telephone and Internet capabilities, the Center was also one of the early TIG grantees. In 2008, Katherine moved to private practice, establishing a successful unbundled practice supporting self-represented litigants in Alaska, and helped to create the first Unbundled Law Section of a state bar. An early member of Self-Represented Litigation Network, she has led the SRLN since August 2013. Katherine is particularly interested in building delivery systems for rural and vulnerable populations and creating community based legal assistance environments that prioritize judicial engagement and leadership to re-imagine services and approaches used by the private bar, legal aid, court staff and non-legal community providers so that everyone can get the legal help they need, when they need it in a format they can use. Katherine sits on the Advisory Committee for Voices for Civil Justice, serves as the Senior Advisor to the Justice for All Project, is a Non-Resident Senior Fellow at the Georgetown Institute for Technology Law and Policy, and member of the Board of Advisors of the Journal of the National Association of the Administrative Law Judiciary. Katherine, a graduate of Northwestern University and Seattle University School of Law, resides in Virginia.
The 28th Annual CALI Conference for Law School Computing® brings together leading technology professionals, faculty, librarians, and institutional leaders to discuss the transformation of legal education through technology and innovation.
CALIcon18 is June 7 & 8, 2018 at American University Washington College of Law in Washington DC. For details including registration information and list of sessions, visit the CALIcon18 website at http://2018.calicon.org/.
Updates to www.cali.org
Updates to www.classcaster.net
Note: CHANGELOG is an occasional feature letting you know about changes to various CALI websites. It is mostly technical in nature but may include initial listing of new features and resources.
Recently the inimitable Roger Smith wrote a pair of articles referencing conversations we had during the LSC Innovations in Technology Conference that was held in New Orleans in January of 2018. I apologize for the frank, but malapropos expression “sleeping with Google”. Let me expand on my thinking a little more.
According to a 2017 LSC report, 86% of low income people with a legal problem receive inadequate or no professional legal help. That’s a justice gap!
These “non-seekers” either don’t know they have a legal problem, don’t know their problem has a legal solution or don’t trust lawyers. They also might be inhibited by despair, depression, language or disability.
When people do seek legal assistance, one study shows that they do so only 13% of the time. In contrast, they talk to family or friends more than a third of the time. Since some do both (8%), we might say 21% are looking online for assistance. Where do they go?
Search engine market share (visited 1/24/18) shows 3/4 go to Google.
You may have noticed that when you search certain categories of information on Google, you not only get a list of results, you get a special interface that is relevant to the category of information being searched. For example, searching a current movie title will result in movie theatres where it is playing. Searching an unusual word will result in a dictionary definition. Searching historical facts will result in a Wikipedia link in the sidebar. Searching a medical condition will often yield links to WebMD, Mayo Clinic or other relevant and authoritative source.
What if you search a legal problem?
You will often find articles or links to a legal aid website, law firm or court self-help article. The results most often look like what you expect from Google – a list of results. There is no special sidebar or treatment for legal problems. Why not? Can we change this?
Many times I have heard about excellent, innovative legal self-help websites and tools that languish for lack of use. They all struggle to find their online audience. If you have the budget, you resort to IRL advertising on billboards, public transportation, brochures, etc. We are all trying to find the eyeballs anywhere we can. Most legal self-help seekers eyeballs start with Google. The biggest legal-assistance-finding system in the world is Google, so we should start there.
The efforts to create a single source website that triages self-help seekers – whether through AI, expert system or whatever, seems like a rational approach, though it suffers from the same problem that all brand new websites do – finding its audience. Theoretically, if we all shout as loud as possible about this single point of finding everything to do with legal self-help, that will create enough noise to direct SRLs or it will create a large enough presence for Google to rank it high in its search results. The efficacy (and admittedly, the fallacy) of Google is that you get a exactly one empty box to type in some words and then you get a list of answers. Scan, click, happiness! A triage system intercedes to get clarification on just exactly what you are looking for – even if you don’t know yourself. The challenge is whether we can do this in the face of the expectation of the click-and-done culture of the web.
Why should we feed the Google beast? Google is a commercial entity who will doubtless want to monetize the searches and profiles of the very SRLs we are trying to serve. The short answer is the same as when a reporter asked Willie Sutton why he robs banks. His reply was “Because that’s where the money is.” Google (and all of the online media giants) use personalized information like location, socio-economic status and browsing history to categorize us for advertisers.
We can help this along by getting very smart about SEO, using Google Sitemaps and developing our own user-centric vocabulary (I almost said taxonomy) that Google and others can adopt as a way to provide up-to-date information. A good analogy here is how Google came to create the General Transit Feed Specification (formerly Google Transit Feed Specification) so that they could include timings for buses and trains in Google Maps. We need a similar spec for legal aid that improves on previous attempts like the Legal Services National Subject Matter Index.
Maybe the Microsoft-LSC-Probono.net Triage Project is covering this ground. They will certainly need some kind of taxonomy to organize the myriad legal problems that people will expect it to address. It’is also an opportunity for Bing (7.92% search engine market share) to close the gap on Google. I don’t know if anyone from the Bing team is on this project, but it would be a really good idea.
The goal here is to get SRLs to the best possible resource for their legal problem, location, economics, and ability to manage it. If triage is a giant tree with branches, these are the leaves. There need to be tens of thousands possible endpoints that are useful to 100% of the searchers as mandated by the Report of The Summit on the Use of Technology to Expand Access to Justice. These could be articles, checklists, A2J Author Guided Interviews, HotDocs templates, agency referrals, legal aid intake portals, videos, etc.
There is still a lot of work to be done creating these resources that a national triage system will deliver up. Under an LSC TIG grant, CALI and Chicago-Kent College of Law examined the landscape of automated forms and found that less than 20% of the most requested or popular forms have been automated.
Everything has been automated somewhere, nothing has been automated everywhere across all 50 states.
We know that automating forms are a successful and popular action from the thousands of user testimonials at the New York Family Court’s DIY forms website. 
Over 100,000 people a year use A2J Author Guided Interviews and are surveyed about their experience. Thousands have responded positively. Go ahead and read some these, they will warm your heart.
Maybe we don’t have to sleep with Google or Bing/Microsoft but we should certainly swipe right and see if they want to get coffee sometime. Search engines are the primary way that the public seeks legal information. Every single website or service we build will have to fight to get the attention of very distracted eyeballs. Why not find those eyeballs where they start searching?
John Mayer
Executive Director
Center for Computer-Assisted Legal Instruction / CALI
jmayer@cali.org
@johnpmayer
Links in this Article
‘Let’s Sleep with Google’: an interview with John Mayer
https://law-tech-a2j.org/odr/lets-sleep-with-google-an-interview-with-john-mayer/
Sleeping with Google: what might that mean?
https://law-tech-a2j.org/legal-market/sleeping-with-google-what-might-that-mean/
The Justice Gap: Measuring the Unmet Civil Legal Needs of Low-income Americans https://www.lsc.gov/sites/default/files/images/TheJusticeGap-FullReport.pdf
Legal Marketing Stats Lawyers Need to Know – Thursday, October 1, 2015 – https://www.natlawreview.com/article/legal-marketing-stats-lawyers-need-to-know
The Justice Gap: Measuring the Unmet Civil Legal Needs of Low-income Americans – https://www.lsc.gov/sites/default/files/images/TheJusticeGap-FullReport.pdf
NetMarketShare.com – https://netmarketshare.com/search-engine-market-share.aspx
IRL definition – http://www.dictionary.com/browse/irl
I Rob Banks Because That’s Where the Money Is
https://quoteinvestigator.com/2013/02/10/where-money-is/
The Legal Services Corporation Announces Pilot States for Innovative Program to Increase Access to Justice – LSC, Microsoft, and Pro Bono Net to Partner with Alaska and Hawaii to Create Statewide Legal Access Portals
https://www.lsc.gov/media-center/press-releases/2017/legal-services-corporation-announces-pilot-states-innovative
Report of The Summit on the Use of Technology to Expand Access to Justice
https://www.lsc.gov/media-center/publications/report-summit-use-technology-expand-access-justice
New York State Courts Access to Justice Program – User Testimonials
https://www.nycourts.gov/ip/nya2j/diytestimonials.shtml
Definition of “swipe right”
https://en.oxforddictionaries.com/definition/swipe_right_%28or_left%29
Learn About Google Sitemaps
https://support.google.com/webmasters/answer/156184?hl=en
Google Transit: How (and Why) the Search Giant is Remapping Public Transportation
https://www.xconomy.com/san-francisco/2012/02/21/google-transit-a-search-giant-remaps-public-transportation/?single_page=true
The National Subject Matter Index: Categorizing Information for Poverty Law
https://lsntap.org/LSXML_NSMI
CALI QuizWright™ is a web app that lets law faculty write individual Multiple Choice, True/False, and Yes/No questions, saves the questions in a personal question bank, builds quizzes with the questions from the bank, uses CALI AutoPublish to instantly and securely publish the quizzes to the CALI website and lets students take the quizzes as formative assessments, either live in class or as homework, right on the CALI website, where most students already have an account. Faculty can view the basic results or access advanced analytic reporting from an online dashboard. Download of results to Excel for further analysis is available.
Our goals in creating QuizWright are to give faculty a straight forward way to create and distribute formative assessments for their students that doesn’t require the installation of additional software, to provide a tool that is intuitive to use and doesn’t require extensive training for faculty, to leverage proven CALI tools and resources that are already familiar to most students and many faculty, and to make legal education better.
The place to get started is www.cali.org, the CALI website. Log in with your faculty level account and you’ll have instant access to QuizWright. No account? No worries! Virtually all US law schools are CALI members and faculty, librarians, and staff at member schools have full access to a wide range of CALI resources and tools including QuizWright. To register and create an account you’ll need your school’s authorization 
code. Check out the CALI law school contacts page to find out who to ask for the authorization code at your school. With the code registration just takes a minute and then you’ll have access to a wide array of CALI resources and tools
Once you’re logged in to the CALI website you can find the link to QuizWright in the CALI Dashboard dropdown menu at the top of the page. Clicking the link will take you right into QuizWright.
Before you get started creating questions and quizzes we 
suggest you take a minute to create a profile on QuizWright. Click on the My Profile link in the left hand menu of QuizWright. The information you enter in this profile is included in the quizzes you create, appearing on the title page of the quiz.
To start adding questions just click on New and then Question in the left menu. The question creation process starts with a choice of question type. QuizWright handles True/False, Yes/No, and Multiple Choice questions. For T/F and Y/N questions you enter the text of the question and select the correct response. For MC questions you enter the text of the question, the correct choice and up to 3 additional incorrect responses. Optionally you can enter brief feedback for the student that is displayed after the question is answered.
We strongly suggest you write out the questions and answers before you fire up QuizWright so you have something to work with. Once you’ve entered a question you can use the buttons at the bottom to either Add another question or Prepare the Quiz. Questions are saved in your personal question bank.
Once you’ve entered the last question you want to include in your quiz hit the Prepare the Quiz button to bundle your questions into a quiz. You’ll see a listing of your questions. Check the one’s you want to include in your new quiz and hit the Add a new quiz button to open a new Quiz page.
Add a title and brief description in the appropriate fields on 
the Quiz Information form. This will be displayed on the quiz’s page on the CALI website and included in the quiz itself. Optionally you can add some text for introduction and conclusion pages. These will be shown at the beginning and end of the quiz if you chose to create them. Finally hit the Save Quiz button.
Saving the quiz takes you to the Quiz Detail page. You can save and edit the quiz from this page, but we’re interested in the Publish Quiz button. Click on that button to publish your quiz to the CALI website. Once that button is clicked you are taken to the CALI website, your QuizWright quiz is published and it’s ready for your students to take. You can get the link your students need to take the quiz by clicking on the View tab at the upper right of the edit page you landed on when the Quiz was published and copying the address from the address bar of your browser or you get the link from the email that is sent to you when you publish the quiz. The URL for your quiz should look something like https://www.cali.org/aplesson/17044 where the number part at the end is unique to your Quiz.
Once your students have the link they can click on it and take the quiz like they would 
any CALI Lesson. They will need to have a valid account on and be logged in to the CALI website in order to take a QuizWright quiz. You can see students results in real time by following the link yourself and launching the quiz. Your special faculty view of the Quiz allows you to see student results in real time. You can review results later by clicking on the AutoPublish link in the CALI Dashboard dropdown menu at the top of the page.
And there you have it, Zero to Quiz in 5 minutes (or so) with just 5 quick steps to follow. QuizWright has a lot more features for authoring and managing questions and quizzes but this brief overview should be enough to get you started working with formative assessments and CALI QuizWright. If you have any questions about QuizWright or any other CALI tools or resources check out the CALI FAQs or drop me a note at emasters@cali.org.
Note: CHANGELOG is an occasional feature letting you know about changes to various CALI websites. It is mostly technical in nature but may include initial listing of new features and resources.
location and publishdate fields to the data model
