2015 ACM-ICPC North America Qualification Contest

Contest results

Here is the problem set and here are the Judge's data.

We had 1221 teams register on ICPC registration, but 254 of these had no contestants added, so we effectively had 967 teams that could show up.

We had 977 users on 702 teams sign in, and there were 642 teams that submitted something. Of those, 609 teams (or 94.8% of teams submitting) solved at least one problem. There were 6733 total submissions, or an average of 22.4 per minute for 5 hours.

Here's a table of how many teams solved how many problems:

Problems solved	0	1	2	3	4	5	6	7	8	9	10	11
Number of teams	364	36	66	158	86	84	50	40	23	16	23	27

Recall that some teams were individuals, others had multiple members, so it is not reasonable to compare the performance of teams having different numbers of members.

Here are stats on the problems:

	A	B	C	D	E	F	G	H	I	J	K
Solved / Tries	262/793 (33%)	264/641 (41%)	78/162 (48%)	101/385 (26%)	50/141 (35%)	565/973 (58%)	116/441 (26%)	503/872 (57%)	597/942 (63%)	228/626 (36%)	77/197 (39%)
Average tries	2.35	1.87	1.53	2.21	1.83	1.66	1.95	1.62	1.54	2.26	1.86
Averages tries to solve	1.95	1.63	1.38	1.92	1.48	1.58	1.60	1.46	1.46	2.07	1.39

Here's the distribution of the delay in judging responses from Kattis:

Maximum delay (seconds):	10	20	30	40	50	60	70
# of submissions:	5639	1040	185	33	13	6	1

Here is the rate of submissions over 10-minute intervals during the contest:
Rate of submissions during the contest

Thanks

This contest went extremely smoothly overall. Many thanks go to the following people:

Those who wrote problems: Chad Mourning, Christopher Raphael, David Sturgill, Godmar Back, Jeff Kinne, Joshua T. Guerin, Kathleen Ericson, Michael Goldwasser, Nathan Backman, Robert Hochberg, and Xiaoyuan Suo.
Those who reviewed problems: Amanda Holland-Minkley, Antonette Logar, Arjun Guha, Ben Cousins, Brian C. Dean, Brian Harrington, Brian Howard, Bruce Elenbogen, Chad Mourning, Choong-Soo Lee, Chris Marriott, Chris Raphael, Daniel Joyce, David Poplawski, David Sturgill, David Wahler, Denis Savenkov, Don Bagert, Elena Machkasova, Eric Harley, Godmar Back, Grant Braught, Howard Cheng, Howard Whitston, Ivor Page, Jay Urbain, Jeff Ifland, Jeff Kinne, Josh Guerin, Kathleen Ericson, Kathy Ames, Keith Edwards, Kevin Wortman, Lee Wittenberg, Liam Keliher, Lily Chang, Manki Min, Mario Nakzawa, Michael Goldwasser, Michael Thompson, Nathan Backman, Nathan Gossett, Robert Hochberg, Robert Tairas, Robert Trenary, Ross Sowell, Sat Garcia, Scott Kerlin, Sean Deitz, Troy Vasiga, Uttam Thakore, Victoria Eisele, and Xiaoyuan Suo.
The judges: Andrew Harrington, David Sturgill, Edward Corwin, Joshua Guerin, Kathleen Ericson, Michael Goldwasser, Fredrik Niemelä, and Xiaoyuan Suo
The ICPC registration team, in particular Jeff Donahoo, Jan Kubicek, Lisa Donahoo, and Marsha Poucher.
The Kattis team: Fredrik Niemelä, Pehr Söderman, Gunnar Kreitz, Per Austrin, and Mikael Goldmann.

Key information for the contest

Where to login for the contest: https://naq15.kattis.com/
How to login: with your email address used in the ICPC registration system.
Having trouble? Please read the information on this page, and if you can't find an answer, ask on IRC at #naq2015 on quakenet.
Do you have a clarification request about the problems? Please read them carefully first, and then if needed ask on Kattis (see the "Clarifications" link at the top of the page).

Kattis accounts and Kattis teams created — We pulled the latest data from ICPC registration and created the Kattis accounts and teams for the contest. Please login to https://naq15.kattis.com/ for the contest using the same contestant email address that is in the ICPC registration system. At this point no modifications to the ICPC registration system data will make it into the contest; you will need to contact contest directors for such help (and it may not be possible to accommodate all requests).

The fourth ACM-ICPC North America Qualification Contest will be October 3, 2015 at 11:00 - 16:00 (CDT). The contest will be held online (on Kattis). If you are a coach, please register your teams (see below). If you are a team member, please communicate with your coach to get your team registered.

Registration (CLOSED)

Registration is now closed. If you (as a coach) have registered teams on icpc.baylor.edu for the North America Qualifier, then you are done. If you have missed registration, we apologize but no additional teams may be registered at this point due to time constraints. However, you can and should populate all your teams with the participants who will be on the teams, so that we may create their Kattis teams.

Each year we have had people register for the incorrect contest, and that has prevented people from participating. Please ensure you have registered for the contest titled "North America Qualifier". Follow the instructions and links below.

There is one contest, with 12 "sites". The 12 sites correspond to the 11 regions of North America, plus one extra site for contestants who are not eligible to compete in ICPC in North America (e.g. high school students or people whose school is not in an ICPC North America region). Please register for the site that corresponds to your region. Here are direct links for each region (using these is likely the simplest route to registration):

You may register as many teams as you like (each team can have anywhere from 1 to 10 competitors — that's up to you). If you are in doubt about how many you may need, register more up front (as it will be impossible to add teams after registration freezes). But please try to be somewhat realistic for our planning purposes.

After registration is closed, due to the number of teams and the limited support it will be impossible to accommodate new registrations or changes to existing registrations. So please register early, make sure you are registered correctly and for the correct contest and site. Note that you should register for the contest called "North America Qualifier". There are other contests with the word "Qualifier" in the name, so be careful. Registering for your regional contest does not register you for this contest.

Once registration is closed, you will receive information on how to access the contest control system (Kattis) via email to the registered coach of each team.

Basic details

What is it? An online (distributed) programming contest, offered as a drop-in replacement for so-called "qualifying" contests (e.g. school-level, pre-regional competitions).
When is the contest? October 3, 2015, 11:00 - 16:00 (CDT).
When does official registration begin? During the summer.
When does official registration close? Please see the official ICPC registration site.
Can I register after registration is closed? No, due to the number of expected people participating and the limited support. Please make sure to register for the correct contest before the deadline.
Where do I register? See the links above on this page.
What contest should I register for? The site corresponding to your region. Note that this is the "North America Qualifier", not a regional competition. As the names of different contests may look similar, please make sure you register for the North America Qualifier, and the site corresponding to your region.
What if I am not a part of ICPC? Please register then for the ICPC-Ineligible site. This is for contestants who are (e.g.) in high school or are otherwise ineligible to participate in a North American ICPC region.
Is this a qualifying contest, a practice contest, or something else? We are calling this a qualifying contest for the regional competitions. It can be (but is not required to be) used to help choose teams for the regional programming competition.
Why is it being offered so early in the fall? To make sure it precedes all North American regional competitions.
Must students participate in teams of three? You may make teams of one to 10 contestants, but no prescribed number is required. You decide for your school and teams.
Must I participate? No. If you don't find this useful for your situation, feel free to ignore it.
I already offer a contest. Why should I use this one? There are several benefits. It will save you work, increase registration for your region (which helps in allocation of wildcard slots for the world finals), and gives all registered students a free year of ACM student membership.
How will results be used? However the coaches choose to use them. The results do not dictate anything about which contestants get to go to regionals, for example. However, one way we envision a coach may use the results is to select the top-finishing team(s) for the regional competition.
Where should I go to compete? You compete wherever you are. This is an online competition, so teams can meet in a common location, but you should organize that on your own.
What resources does a local site need to provide? For each team, a computer with internet access, a web browser, and whatever development environment(s) you wish.
What judging system will be used? Kattis. You can create a free account and try out Kattis at open.kattis.com. Note that the actual contest may be hosted at a different website with different login credentials (which your coach will receive via email after registration closes).
Do I need to provide / can I provide my own judging? No, it will be done through Kattis and our centralized contest management.
Can I provide my own problems? We welcome your offer to contribute problems to the official problem set — if you wish to do this, please contact Greg Hamerly. However, we will not support additional problems for just one site.
What languages will be supported? C, C++, and Java.
How many problems will there be? About 10.
What will be the difficulty level of the problems? We are aiming for a wide range, with multiple easier problems for novice programmers.
How will the contest run? Contestants will submit code and receive responses via the web. Coaches/facilitators should ensure the integrity of their own contestants (such as internet usage during the contest, appropriate books and materials, etc.).
Is this one contest or multiple contests? While we will use one set of problems, offered at one time, each participating school will be in its own contest with its own set of results. Results will not be collated across schools.
How much does this cost? It's free.
Will there be a practice portion of this contest? You can practice using the Kattis system at open.kattis.com. You can create a free account and submit problems to the judging system, just like in the qualifier. The actual qualifier will be hosted at a different website with different credentials.
Who should I contact with questions? Greg Hamerly

Contest scoring

This contest scoring system will be the same as the world finals. That is, the winner is the team solving the most problems. If two teams solve the same number of problems, then the team with the lowest time is the winner. If two teams have the same time the submission time of the last solved problem is used as a tie-breaker.

The time is the sum of the time of submission (in minutes) of the earliest correct submission for each solved problem, plus any penalty minutes for each incorrect submission of a problem prior to solving that problem. Penalties are 20 minutes for any of the following reasons:

Run Time Error
Time Limit Exceeded
Wrong Answer
Output Limit Exceeded
Memory Limit Exceeded

Compile error does not incur a penalty; it is not considered a valid submission. Illegal Function does not incur a penalty, but any illegal function will be investigated and a team may be disqualified from the competition if the judges consider the program an attempt to exploit the contest system.

For more information on how Kattis scores problems, please see Kattis documentation.

For problem-writers

We are using the Kattis Problem format for developing problems. This is the format used for the ICPC world finals. It provides a structure for problem writeups, test data, solutions, and configuration. It automates problem verification (for solutions and anti-solutions), time limit determination, and finally installation. There are two parts: the problem format and the problem tools.

Kattis problem format

Here is some information on the Kattis problem format:

There is a very detailed guide to developing problems in the Kattis problem format.
The minimum things you need to provide for a problem are:
- Read and fill in the relevant problem.yaml fields — at least "author". Most are done for you already.
- Write the problem statement in problem_statement/problem.tex using simple LaTeX.
- Put sample and secret test data files in data/sample and data/secret using .in and .ans suffixes with corresponding prefixes (e.g. A.in corresponds to A.ans).
- Put at least one solution to the problem in submissions/accepted (preferably one in Java and one in C/C++)
Other nice things to have in a problem are:
- submissions/wrong_answer — program(s) which give wrong answers on (some) included test data
- submissions/time_limit_exceeded — program(s) which take too long on (some) included test data
- submissions/run_time_error — program(s) which cause a run time error on (some) included test data
- data/*/*.desc files to describe each test (e.g. A.desc)
Start by modifying this sample ("skeleton") problem package.
For more examples of problems formatted in the Kattis problem format, here is the problem set for the 2014 ICPC NWERC. Warning: this is a large download; about 90MB.

Kattis problem tools

After you're done writing your problem, you can use the Kattis problemtools software on a Linux machine to verify the problem (using "verifyproblem") or to compile a PDF or HTML version of your problem statement (using problem2pdf or problem2html, respectively). You can get the Kattis problemtools in two ways:

Try first to install this precompiled Kattis problemtools package for Debian / Ubuntu on your Ubuntu or Debian Linux distribution. This was built on Ubuntu 14.04 (trusty).
If that doesn't work, you can download and build the Kattis problemtools package from the source on Github. Clone or download it, run "make builddeb", and then install the resulting package (which is placed in the parent directory).