Digital India Awards 2016 used a rigorous and scientific methodology to determine the 28 winners across 8 different categories of awards including Special Jury Awards.

The universal acceptance of the power of ICT in transforming and accelerating the development process, especially in developing economies, is indisputable. Government of India has also been proactively engaged in the seamless delivery of information and services with the adoption of best ICT practices. In order to promote more innovative e-Governance initiatives by Government bodies, Digital India awards (earlier known as the Web Ratna Awards) were instituted under the ambit of National Portal of India. The award acknowledges exemplary initiatives of various government entities in the realm Digital India.

AWARD CATEGORIES Considering the vision of Digital India and acknowledging the fact that citizen are now accessing government information and services through media other than the web including, but not limited to, Smartphone Apps, the scope of the Web Ratna awards (2014) was broadened to include these initiatives in addition to the other categories. Digital India Awards 2016 have been constituted under the following eight categories: • Exemplary Online Service • Innovative Citizen Engagement • Web Ratna - Ministry/Department • Web Ratna - State/UT • Web Ratna District • Outstanding Digital initiative by Local body • Best Mobile App • Open Data Champion

ONLINE NOMINATION Online nominations had the following parameters:

General Parameters • Contact details of the team leader and team members, details of the website like Title, URL, overview etc. • The salient features like Usability, Interface, Outreach, Bilingual interface, Content personalization features, Presence of contact details, Feedback form/Contact details, FAQ and Help.

Category Specific Parameters Most Innovative Citizen Engagement: Relevance, target audience, modes of participation, extent of participation and outcomes. Exemplary Online Service: Service maturity (informational, enhanced informational, transactional or connected), convenience, transparency, number of feedbacks received/redressed, number of grievances received and addressed. Web Ratna - Ministry/Department: Details of major e-Governance initiatives covered, reach, stakeholders, subordinate departments/offices that have their own websites, convenience in availing of information and/or services, number of feedbacks received/redressed etc. Web Ratna - State/UT Government: Spectrum of coverage, number of State Departments with their own websites, number of transactions to complete a task, time taken per transaction, convenience to avail information/services, no. of feedbacks received/redressed etc. Web Ratna - District: Number of Online Services & Certificates, information for local residents/tourists, convenience, transparency, cost effectiveness, efficiency, enhancement etc. Outstanding Digital initiative by Local body: Number of Departments associated, content, online service quality, service maturity, convenience to avail information/service, number of grievances received/addressed, etc. Best Mobile App: Relevance, ease of use, technical, App functionality, privacy concerns, social media integration, incorporation of analytics functionality, feedbacks and suggestions, etc. Open Data Champion: Presence on Data Portal of India (http://data.gov.in), compliance with the National Data Sharing and Accessibility Policy (NDSAP), description, usefulness, availability for statistical inferences, etc.

EVALUATION PROCESS Extensive process for evaluation of filed nomination was followed which was conducted in two stages.

Stage-1: Entries were screened and evaluated by IIT Delhi based on a scientifically formulated methodology.

Stage-2: The distinguished Jury under the Chairmanship of Secretary, MeitY and experts from Academia and Industry, thereafter reviewed the shortlisted nominations and finalized the awardees in each category. Three awards were presented in each category namely Platinum, Gold and Silver along with Jury choices in few categories. A total of 28 entries were selected to receive the Awards.

INSTRUMENT DESIGN FOR EVALUATION A proficient team from the Indian Institute of technology (IIT), Delhi played an active role in screening the applications based on a scientifically devised process.

Each category was assigned with some specific features which a entry had to qualify to get shortlisted for the award. Criteria were defined for each category based on their specific and regular features. for example for the best mobile App the criteria can be seen in figure 3.

For each criteria, a set of questions was prepared. The questions were mapped with website/App features either based on yes or no (0 or 1) score, or based on Likert scale (0-5).

Questions for criteria ‘Ease of Use’ for ‘Best Mobile App’’ are given in Table-1. Similarly, questions were prepared for each of the other criteria for this category.

Weights Assignment The multi-criteria approach Analytic Hierarchy Process (AHP) was used to assign the weights to each of these criteria. AHP was developed by Thomas L Saaty in 1980. AHP is a rational framework for structuring a decision problem, representing and quantifying its elements, relating those elements to overall goals and evaluating alternative solutions.

Analytic Hierarchy Process (AHP)– Inputs • Relative importance of criteria • Through pair-wise comparison matrix • Expert opinion to assign the weights for each question W1, W2, W3….Wn in respective questionnaires.

To arrive at the weights first a pair-wise comparison matrix was created. In this, the various criteria were compared in pairs for importance on a 9 point scale as given in Table-2.

Based on the comparison matrix in Table-3, the weights were arrived after normalising and performing a consistency check. The final weights for the criteria for best mobile app are given in Table-4. Security audit and certification were mandatory for any entry to be considered for evaluation.

Data Collection The data was collected for each nomination based on questionnaire instrument of that category (Table-1). Double blind assessment was followed for ascertaining impartial outcomes. Third evaluation was done if needed.

Results • Each entry was rated against each criteria c1, c2, c3…Cn. • Final score was calculated as following: • Weighted score = C1W1 + C2W2+ C3W3+ ….+CnWn Nominee entry with highest weighted total sum was declared as winner under each category.

SUMMARY The methodology used for selecting the winners of Digital India Awards 2017 was extensive, scientific and quantifiable, ensuring above par decisions and fair analysis. This model can be adopted for evaluation for similar requirements in Government.