Project Sunflower: Time to Launch Application, Open a Book and Flip Page
Well, we now have results regarding the time taken for the Apple iPad 2, Amazon Kindle DX and Motorola XOOM to render eBooks. We installed iBooks on iPad and the Kindle App for Android on XOOM. The Google Books app can’t be installed (yet) in UK due to copyright issues. We recorded the time taken by the devices to open the app, open a book, and flip a page.
Since there is no emulator that performs exactly like the physical device, we chose to take a practical approach to measure the times. The render times were measured in two ways. One, manually, and the other using a video camera.
Take a stopwatch in one hand, and have the other hand tap on the device. For example, when using an iPad, what we did was to hold the stopwatch in the left hand, and tap the iPad with the right hand. Start the stopwatch precisely when the iPad is tapped, and stop when the desired action is done. This method depends a lot on the user’s reflexes and you may have your doubts about the level of precision when it comes to results. Let me tell you, the results were surprisingly accurate. Read the figures to see for yourself.
This is a slightly more sophisticated way of measuring, thouukolngh just as simple. All you need to know is the fps (frames per second) at which the video is recorded, and a video player that can replay the video frame-per-frame. Record the desired action on-camera, and then replay the video frame-per-frame. The number of frames traversed from the start to end of the task gives a more precise time taken to complete the task than the manual method.
We measured the times for six free eBooks per device. Six readings were taken per task, and the average time for each task was calculated.
iPad (Average from six readings)
Kindle DX (Average from six readings)
There is no application load time as all the books are displayed directly on the Homescreen.
XOOM (Average from six readings)
Both the methods gave fairly similar results. The differences in the times on an average are:
The standard deviations for each method are shown below:
iPad (Standard Deviation)
Kindle DX (Standard Deviation)
XOOM (Standard Deviation)
The standard deviation tables show that the camera method showed less variation from the average as compared to the manual method in all but two cases, where the difference is only 1/100th of a second. These two cases may be safely ignored.
Although both methods gave fairly similar results, it must be noted that the manual method would give varied results for every test. It is completely dependent upon user reflexes, and slow reflexes could have seriously bad results. The camera method does take up more time, however the results are more accurate and dependable So, I’d recommend the camera method.
All the recorded times are an average, and the times may change with the length of the books proportionally. These results give us a fair idea about the various devices when it comes to render speed and page flipping. The iPad and XOOM clearly render faster than Kindle DX. However, these results are only pertaining to the device capabilities and say nothing about the user experience. What makes an eBook reader good or bad does not depend only on the render speeds, but more so on the user experience the device has to offer. A detailed usability study of the devices will be undertaken soon which will shed light on the varied user experience, and help us better understand what the user expects from an eBook reader.