Urban Plans Assessment

The study was focused on the evaluation of cartographic quality of urban plans in the Czech Republic using eye-tracking. Although map visualization is a crucial part of the urban planning process, only a few studies have focused on the evaluation of these maps. The plans of four Czech cities with different styles of visualization and legends were used in this eye-tracking experiment. Respondents were required to solve spatial tasks consisting of finding and marking a certain symbol on a map. Statistical analyses of various eye-tracking metrics were used, and the differences between experts and students and between the map and legend sections of the stimuli explored. The study results showed that the quality of map symbols and the map legend significantly influence the legibility and understandability of urban plans. For correct decision-making, it is essential to produce maps according to certain standards, to make them as clear as possible, and to perform usability testing on them.

Methods

 Stimuli

This study considers urban plans published using web applications. To analyze comparable urban plans, it was necessary to choose applications with similar layouts (large map, a legend on the right hand side, no additional map features). Four urban plans created by different authors using different styles and published in different years were selected. The oldest plan is that of Jihlava (1999—revised in 2013). The Hradec Králové plan was created in 2012, and the Bohumín and Olomouc plans were created in 2014. All the plans were created by private companies; each company adopted their approach to visualizing the urban plan (no methodology described in the introduction was used). The tasks were designed as static views, and it was therefore not possible to move the maps. The advantage of this approach is that the maps had similar scales and eye-tracking data recorded for all respondents and their answers could be directly compared. Nevertheless, it was impossible to use images as stimuli in the eye-tracking experiment. The legend for all the maps was longer than the height of the image. We created an HTML page for each city/task consisting of the image with the static map on the left and a legend in a window with a slider on the right. The results look exactly same as the original web pages with the plan, and although it was possible to move the legend, the map was static. The same zoom level (scale 1:5000) was used for all the plans because the printed version of each plan in this scale is also available. In this scale, only a small part of the city was displayed on the screen, which was useful to eliminate any participant knowledge of the cities.

 Tasks

To cover the most typical tasks during standard work with an urban plan (i.e., finding a new place for housing or identifying areas with proposed public services), analysis of the urban plans was based on six tasks. To prevent any misunderstandings, the tasks were created to be as simple as possible. The tasks focused on the point, line and polygon map features to cover as many cartographical symbols as possible. The following questions were asked:

• Q1: Mark an area for housing (Zoning map)
• Q2: Mark an area for sports or recreation (Zoning map)
• Q3: Mark an area for proposed public services (Zoning map)
• Q4: Mark a railroad (Transportation map)
• Q5: Mark a wastewater treatment plant (Utility networks map)
• Q6: Mark a protected area of water resources (Natural protection map)

 Participants

Thirty-four respondents participated in the experiment. Because of the inaccuracy of calibration or an insufficient Tracking Ratio, eight were excluded from the results analysis. Finally, we obtained 26 respondents. Twenty of them were students (11 male and 9 female) in the third year of the Geoinformatics and Cartography bachelor’s program or the Geoinformatics master’s program. Six respondents (1 male and 5 female) were experts working in the departments of urban planning at the Municipality of the City of Olomouc or the Regional Authority of the Olomouc Region. These experts worked with urban plans every day and had similar backgrounds.

Apparatus

Eye-tracker SMI RED 250 (developed by SensoMotoric Instruments, Berlin, Germany) with a sampling frequency of 250 Hz was used in the study, and the experiment was created at the SMI Experiment Centre. Stimuli were presented on a monitor with a resolution of 1920×1200 px

Procedure

At the beginning of the experiments, the respondents completed a short questionnaire about their age and experience, and then calibration was performed. We set a calibration threshold of 1° of the visual angle. Respondents with a higher deviation were excluded from the results, along with those whose Tracking Ratio (the proportion of time that the eye tracker recorded point of gaze coordinates over the entire experiment) was higher than 90 %. After calibration, respondents were informed about the purpose of the experiment and provided with some basic information about urban planning. The tasks were then presented. No time limits were set for respondents to read and remember the task. After pressing the F2 key, an Internet browser would open, displaying a web page with one of the stimuli automatically. The tasks were presented sequentially from Q1 to Q6. The city plans were randomized for each task. At the end of the experiment, a short questionnaire about the respondents’ subjective opinions on the presented plans was displayed. This questionnaire was created in Google Forms and contained an image of the representative plan for each city (to remind the respondent how it looked) and a scale from 1 (best) to 5 (worse). The experiment was ten to fifteen minutes long. A diagram showing the experiment’s design is in Figure 3.

 Analyses

The first part of analyzing the recorded eye-movement data focused on the Trial Duration metric. This metric shows how long it took to solve a task. Higher values of Trial Duration are expected for more complex tasks or plans with lower legibility. In these cases, participants may have problems finding the proper symbol in the legend or identifying it on the map. Statistically significant differences between tasks for each task according to the Kruskal-Wallis test are shown in the upper part of the figure. From the boxplot, it is evident that some tasks were much more difficult than others. Participants were very fast (below 20 s) in tasks Q1, Q2, and Q4. The most problematic tasks were Q3 and Q6. For task Q5, the only problematic plan was for Jihlava.

Areas of Interest (AOIs) were marked around the map and legend of each stimulus. The number of fixations in each AOI were calculated and are shown in Figure. It was difficult to locate the correct symbol in the legend for tasks Q3 and Q6. In Q3, the map symbol for proposed public services was in the legend in the second column, which influenced a high number of fixations in identifying the correct symbol. The most fixations in the legend was observed in Q6 (Olomouc). This result was due to the position of the correct symbol in the legend (in the lower part).

Data from the whole experiment were visualized using a FlowMap method in V-Analytics software. The most illustrative results of a FlowMap method were obtained from the data recorded over the task Q6. As was already mentioned in the methods, FlowMap shows the aggregated eye-movements of all participants. The width of the arrows represents the number of gaze movements between Voronoi polygons in the stimulus. In accordance with previously described results, the most straightforward strategy of stimulus inspection was observed in the case of Hradec Králové. The symbol of a protected area of water resources was quite clear and distinctive, so the aggregated gaze trajectory displayed as a sequence of arrows leads directly to the correct place on the map. The legend was clearly structured, and participants didn’t need too many fixations for the finding of the proper symbol. On the other hand, the most complicated gaze trajectories were observed for Bohumín and Olomouc. The participants spent a lot of time in the legend trying to find the correct symbol in unstructured (Bohumín) or very comprehensive (Olomouc) legend.

In the next part of the data analysis, the accuracy of answers was investigated. The results are summarized in Figure. As mentioned above, the first two questions were easy and consisted of standard task solving while working with the plan. Only one incorrect answer (detected by mouse click coordinates) was recorded.

An interesting result was found for task Q4 (finding the railroad). For each city except Olomouc, all the answers were correct. For Olomouc, three incorrect answers by experts were found. This is interesting, because all the experts work with the Olomouc plan every day. The problem was the black line symbol (at the bottom of Figure 17). Although this symbol can be mistaken for a railroad, in reality it represents a boundary between city districts. Experts were confident of their answers being correct and did not check the symbol in the legend.

Conclusion

Four urban plans created by different authors, having different styles and published in different years were selected. To cover the most typical tasks involved in standard work with an urban plan, the analysis was based on six tasks. Twenty-six respondents (20 students and 6 experts working in urban-planning departments) participated in the study.

We conclude that two crucial factors influence the legibility of plans and significantly impact the understandability of maps. Those are the quality of map symbology (number of colors, design of symbols, features/layers on the map) and logical hierarchy/structure of the legend. From the results of the eye-tracking data analysis and the subjective questionnaire, the worst plan was the Jihlava plan. It was problematic because of the incidence of incorrect map symbols in the legend. The associativity of the map symbols was also low, and line symbols were too thick. According to the results, the best plans were those of Olomouc and Hradec Králové, which had a clearly structured legend, a low number of map symbols, and clearly distinguished colors.

During the explanatory observation of the different behavior between experts and students, the accuracy of answers was found to be dependent on many factors, such as the position of the symbol in the legend, previous user experience, and self-confidence in the correct answer. To avoid these misunderstandings, urban planners should be aware of quality issues in urban plans. For correct decision-making, it is essential to produce maps according to certain standards, make maps as clear as possible, and perform usability testing on maps.

Outputs

Burian, J., Popelka, S., & Beitlova, M. (2018). Evaluation of the Cartographical Quality of Urban Plans by Eye-Tracking. ISPRS International Journal of Geo-Information, 7(5), 192.