27 April 2008

Guide to integrating Stata and external text editors

Last updated: 1 July 2015

Download links

Table of contents

1. Introduction
2. Requirements
3. Installation
4. Using the programs
5. Known problems and troubleshooting
6. Acknowledgments
7. History of the programs

1. Introduction

Stata is a statistical package that is available for Windows, Macintosh and Unix operating systems. This guide addresses some shortcomings of the text editor that is part of the Stata package. Stata's Do-File Editor is fully integrated and commands can be executed directly from the editor with a keyboard shortcut or the click of a button. External editors often have a richer set of features but they lack integration with Stata.

This document describes how external text editors can be integrated with Stata in a way similar to the Do-File Editor. Two free programs are offered for this purpose: rundo.exe allows the execution of complete Stata do-files from an external editor, and rundolines.exe allows the execution of selected Stata commands from an external editor. Both programs can be installed and used together or individually. I recommend installing only rundolines because it can serve the same purpose as rundo (see the explanation in section 4).

2. Requirements

  • MS Windows: The programs were tested with Windows XP, Windows Vista, Windows 7 and Windows 8 (up to version 8.1). Macintosh and Unix operating systems are not supported.
  • Stata: The programs were tested with Stata versions 8, 9, 10, 11, 12, 13 and 14 (up to version 14.0, released in April 2015).
  • Text editor: The programs were tested with Notepad++ 4.8.2 and later versions (up to version 6.5.2, the most recent version available in December 2013). They have also been confirmed to work with ConTEXT, Crimson Editor, EditPlus, EmEditor, PSPad, TextPad, TeXnicCenter, and UltraEdit. In addition, previous versions of the programs were reported to work with gVim, Hidemaru, jEdit, SciTE, Vim, and WinEdt.
  • rundo and/or rundolines; as explained in Section 4, rundolines can be used in a similar manner as rundo and the installation of rundo is therefore optional:

Please contact the author at fhuebler@gmail.com if you use the programs with any editor besides ConTEXT, Crimson Editor, EditPlus, EmEditor, Notepad++, PSPad, TextPad, TeXnicCenter, and UltraEdit, so that the list of supported text editors can be updated.

3. Installation

The integration of an external text editor with Stata can be accomplished with three simple steps:

  1. Download and extract the rundo and rundolines programs. As noted in Sections 2 and 4, the installation of rundo is optional.
  2. Edit the INI files.
  3. Find a way to call rundo.exe and rundolines.exe from the text editor with a keyboard shortcut, a menu entry, or an icon in the toolbar.

Step 1: Download the rundo and rundolines programs and save the contents of the two ZIP files anywhere on your computer. A suggested location is the personal ADO directory of Stata, usually C:\ado\personal\. Each ZIP archive contains four files: the executable, the source code, an INI file, and a readme file.

The programs for Stata 14 were written and compiled with AutoIt version 3.3.12.0. The programs for Stata 13.1 and older versions of Stata were written and compiled with AutoIt version 3.2.12.1. Users who prefer to compile the programs themselves can do so with the included source code.

Step 2: Edit the INI files for the rundo and rundolines programs. rundo.ini and rundolines.ini are identical and list six options that must be edited to match the user's setup.

  • statapath is the path to the Stata executable, enclosed in quotes, for example "C:\Program Files\Stata13\StataSE.exe" or "C:\Program Files (x86)\Stata14\StataSE-64.exe".
  • statawin is the title of the Stata window, also enclosed in quotes, for example "Stata/SE 13.1" or "Stata/SE 14.0". The window title is shown in the center or left corner of the title bar of the main Stata program window (see Figures 1 and 2). Any text that follows the version number should be ignored (for example "- [Results]" in Figure 2).
  • statacmd is the keyboard shortcut to the Stata command window. In Stata versions 8 to 11, the shortcut is Ctrl+4 (written as "^4" in the INI file). In Stata 12, released in July 2011, the shortcut was changed to Ctrl+1 (written as "^1" in the INI file); the same shortcut is used in Stata 13 and 14.
  • clippause is a pause after Stata commands are copied to the Windows clipboard.
  • winpause is a pause between window-related operations, for example between the switch from the text editor to Stata.
  • keypause is the pause between individual key strokes that are sent to Stata.

The last three options are specified in milliseconds. The default values should work for most users but they can be reduced for a faster execution of the rundo and rundolines programs. If the pauses are too short, the programs fail and no commands are passed on to Stata.

Figure 1: Stata/SE 14.0 program window in Windows 8.1
Stata program window with title in title bar

Figure 2: Stata/SE 13.1 program window in Windows XP
Stata program window with title in upper left corner

Step 3: Find a way to call rundo.exe and rundolines.exe from the text editor with a keyboard shortcut, a menu entry, or an icon in the toolbar. This example shows how the programs can be linked to Notepad++, a free text editor for MS Windows. For Stata users, an optional language file for syntax highlighting is available on the Notepad++ Wiki. Other text editors can also be integrated with Stata if they provide an option to call the rundo and rundolines programs; many editors have this capability.

  1. rundolines:
    • In Notepad++, go to the Run menu and select Run.
    • Browse to rundolines.exe and enclose the entire path in quotes. For example, if rundolines.exe is saved in the directory C:\ado\personal\, the complete entry in the Run box of Notepad++ would be:
      "C:\ado\personal\rundolines.exe"
    • Click Save.
    • Enter a name, for example "Run do-lines", and select a shortcut, for example F9 (see Figure 3). Shortcuts with a Shift or Ctrl key should be avoided (see Section 5).
    • Click OK.
    • Click Cancel.
  2. rundo:
    • In Notepad++, go to the Run menu and select Run.
    • Browse to rundo.exe and enclose the entire path in quotes. Next, add the text "$(FULL_CURRENT_PATH)" (with quotes) after the path to rundo.exe. For example, if rundo.exe is saved in the directory C:\ado\personal\, the complete entry in the Run box of Notepad++ would be:
      "C:\ado\personal\rundo.exe" "$(FULL_CURRENT_PATH)"
    • Click Save.
    • Enter a name, for example "Run do-file", and select a shortcut, for example F10 (see Figure 4). Shortcuts with a Shift or Ctrl key should be avoided (see Section 5).
    • Click OK.
    • Click Cancel.

Figure 3: Notepad++ shortcut setup for rundolines program
Notepad++ dialog to add the rundolines program

Figure 4: Notepad++ shortcut setup for rundo program
Notepad++ dialog to add the rundo program

4. Using the programs

After the programs have been successfully installed, Stata commands can be executed directly from the text editor. In the case of Notepad++, press F10 (or any other assigned shortcut) to run a saved Stata do-file. To run only a few commands, highlight the relevant lines in the editor and then press F9 (or any other assigned shortcut). To run an entire do-file with rundolines, select all commands in the text editor (for example by pressing Ctrl+A in Notepad++) and then press F9 (or any other shortcut assigned to rundolines).

The rundo program sends the path of the do-file to the Stata Command window, preceded by the word "do" and followed by Enter. If more than one instance of Stata is open, the do-file is executed in the Stata window that was most recently active. If Stata is not open, the rundo program starts Stata and then runs the do-file. Note that a do-file must be saved before the rundo program is called because it looks for the saved version of the document that is currently being edited. An alternative would be to first select the entire text in the editor and then call the rundolines program, which passes all selected lines to Stata.

The rundolines program saves the highlighted lines in a temporary file and then sends the path of the temporary file to the Command window in Stata, preceded by "do" and followed by Enter. If more than one instance of Stata is open, the commands are executed in the Stata window that was most recently active. If Stata is not open, the rundolines program starts Stata and then runs the selected commands.

Because rundolines can run either a few selected commands or a complete do-file, it can serve the same purpose as rundo. Moreover, the use of rundo can occasionally lead to error messages in Stata 14, which supports Unicode, if a do-file was saved with another text editor (for example the Windows Notepad) and encoded in UTF-8 with BOM because Stata cannot read the BOM character; at the time of writing this was true for Stata 14.0. The author of the programs uses only rundolines and recommends rundolines over rundo. rundo is not necessary and is only maintained for historical reasons.

The rundo and rundolines programs are self-contained, they do not create any files besides a temporary file with Stata commands, and they make no changes to the Windows registry. As the source code shows, the only function of the programs is to copy commands from the text editor and pass them on to Stata. However, both programs overwrite the contents of the Windows clipboard directly after they are executed because the clipboard is used to store the path to the Stata do-file.

5. Known problems and troubleshooting

Users who encounter problems during the installation or execution of the rundo and rundolines programs are advised to follow the troubleshooting steps in this section.

Known problem 1: If the rundo and rundolines programs fail, the clippause, winpause and keypause values may be too small. In that case, edit rundo.ini and rundolines.ini, increase the pause values (especially for winpause and clippause), and then run the programs again. For best results, the values in the INI files are long enough so that the rundo and rundolines programs do not fail and short enough to avoid noticeable delays before the execution of the commands in Stata. The default values in the INI file should work on most systems.

Known problem 2: A second known problem involves keyboard shortcuts to rundo.exe and rundolines.exe that use combinations with Shift or Ctrl, for example Shift+F9 or Ctrl+F10. Such shortcuts can result in a "stuck" Shift or Ctrl key in Stata. This problem affects Notepad++ and other editors. It is therefore recommended to use shortcuts without Shift or Ctrl.

Known problem 3: After an update of Stata, for example from version 13.0 to version 13.1, the rundo and rundolines programs may stop functioning if the title of the Stata window has changed (see Figures 1 and 2). If this is the case, edit the statawin entries in the rundo.ini and rundolines.ini files to reflect the correct window title (for example, replace "13.0" by "13.1").

Known problem 4: When trying to execute rundo, a user can encounter the message "Error: Array variable has incorrect number of subscripts or subscript dimension range exceeded." This problem only occurs if step 3 of the installation instructions was not followed. In this case, please reinstall rundo and confirm that the entry in the Run box of Notepad++ is the following text, including the quotation marks (with a different path to rundo.exe, if applicable): "C:\ado\personal\rundo.exe" "$(FULL_CURRENT_PATH)". An alternative is to use only rundolines, as explained in Section 4.

Known problem 5: Occasionally, the working directory of Stata is changed to the Notepad++ program folder (for example "C:\Program Files\Notepad++") after running rundo or rundolines. This problem occurs when Stata is not already open and is started by calling rundo or rundolines from Notepad++. The solution to this problem is to either start Stata first (before rundo or rundolines are called) or to reset the working directory from within Stata to the desired folder.

Known problem 6: Since Stata 14, released in April 2015, Stata supports Unicode. Rundo and rundolines version 4.1 can be used with Stata 14 but do not support Unicode. Users of Stata 14 should therefore use rundo and rundolines version 5.0, released on 28 June 2015, which support Unicode. Users of older versions of Stata, up to and including Stata 13.1, should use rundo and rundolines version 4.1 because version 5.0 may cause problems with accented characters, such as à, é, ö or ñ.

Known problem 7: Stata 14 supports Unicode but cannot run do-files encoded in UTF-8 with BOM because Stata cannot read the BOM character at the beginning of such files. The Notepad text editor of MS Windows, for example, writes text files with a BOM. If rundo is used to execute such a file the result is an "unrecognized command" error (return code 199) in Stata. The same error occurs if do-files with a BOM are executed from the Stata Do-file Editor. This problem can be avoided by using rundolines instead of rundo because rundolines sends the commands directly from the text editor to Stata.

Troubleshooting: The rundo and rundolines programs should function properly if the installation instructions are followed exactly. If the programs do not work, please use the steps below to test your setup. I recommend testing the rundolines program first because its internal code is less complicated than the code for the rundo program.

  1. Save rundolines.exe and rundolines.ini in the same folder, as described in Step 1 of the installation instructions above. Do not edit and recompile the AU3 file.
  2. Edit the rundolines INI file as described in Step 2 of the installation instructions.
  3. Add a link to Notepad++ as described in Step 3 of the installation instructions.
  4. Start Stata.
  5. Start Notepad++ and type one or more Stata commands, for example "dir" (without quotes).
  6. Highlight the Stata commands in Notepad++ with the mouse or by pressing Ctrl+A.
  7. Press F9 (or any other shortcut that was assigned to rundolines.exe) to call the rundolines program. Stata should now run the commands from Notepad++. If the commands are not executed, the rundolines program cannot find Stata because the window title in the INI file is incorrect. If this is the case, edit rundolines.ini to match the title of the Stata program window, as shown in the installation instructions. For example, note the difference between "Stata" and "Stata/SE" and specify the correct version number of Stata. Save the INI file and repeat steps 4 to 7 until it is possible to run commands from Notepad++ in a copy of Stata that is already open. Move on to the next step.
  8. Close Stata but keep Notepad++ open.
  9. Highlight all Stata commands in Notepad++ with the mouse or by pressing Ctrl+A.
  10. Press F9 (or any other shortcut assigned to rundolines.exe) to call the rundolines program. Stata should now start and run the commands from Notepad++. If Stata does not start, review the rundolines.ini file and make sure that the path to the Stata executable is correct. For example, note the difference between wstata.exe and wsestata.exe. Save the INI file and repeat steps 8 to 10.

When the rundolines program works as expected, the troubleshooting steps can be repeated for the rundo program. However, rundo.exe is more complex than rundolines.exe and may not be compatible with as many editors as the rundolines program. This possible limitation can be overcome by using rundolines instead of rundo as explained in Section 4: to run an entire Stata do-file highlight all text in the editor (usually by pressing Ctrl-A) and then call rundolines.exe.

6. Acknowledgments

The rundo and rundolines programs are based on previous work by Dimitriy Masterov and Eva Poen. Masterov demonstrated in a post to the Stata mailing list how a do-file can be executed from Vi. Poen showed how individual lines can be executed from WinEdt. Nicholas Winter suggested changes to the programs that made them more efficient and solved some problems with the Windows clipboard. Jeffrey Arnold described how INI files can be used to set program parameters, making it possible to distribute precompiled and thus more user-friendly versions of the rundo and rundolines programs.

Others tested the programs with different editors, suggested modifications, or provided general feedback: Alexey Bessudnov, Nicholas Cox, Marcos Delprato, Dirk Enzmann, Matthew Forbes, Ari Friedman, Stefan Gawrich, Masakazu Hojo, Changhwan Kim and Haebong Woo, Keith Kranker, Randall Lewis, Kieran McCaul, Salmai Qari, Tomoyo Sakiyama, Alonso Sánchez, Raul Sanchez, Sven-Oliver Spieß, Timm Sprenger, Francisco Tschen, Renato Vargas, Robert White, Ansgar Wolsing, Amanda Tzy-Chyi Yu, and Markus Zielonka.

7. History of the programs

  • July 2004: First version of rundolines.au3 sent to the Stata mailing list (see Statalist archive).
  • October 2004: First version of this guide, with both scripts, released as part of "Some notes on text editors for Stata users".
  • March 2005: Scripts revised to work with Stata 7 and 8.
  • March 2005: Expanded guide added to my blog on international education statistics.
  • April 2005: Guide and scripts added to my website, with direct download links.
  • 6 June 2005: Version 2.0 of scripts released, with support for Stata 9. The scripts are now compatible with Stata 7, 8, and 9. Scripts work with more than one instance of Stata. Scripts no longer rely on the built-in editor of Stata. New version of this guide.
  • 8 June 2005: Version 2.1 of scripts released. Fixed problem with the clipboard that was reported by some users: scripts no longer attempt to restore clipboard or text in Stata Command window.
  • 30 June 2005: Version 2.1.1 of rundolines script released, with minor bug fix related to clipboard.
  • 26 September 2005: Version 2.2 of scripts released. The scripts are now compatible with Stata 9.1, released on 15 September 2005.
  • 22 June 2006: New section added to the guide: Supported editors.
  • 28 January 2007: The list of supported editors now includes EmEditor, gVim, Hidemaru, jEdit, Notepad++, SciTE, TextPad, Vim, UltraEdit, and WinEdt.
  • 26 April 2008: The following editors have been confirmed to work with the scripts: Crimson Editor, EditPlus, EmEditor, gVim, Hidemaru, jEdit, Notepad++, SciTE, TextPad, Vim, UltraEdit, and WinEdt.
  • 27 April 2008: Version 3 of the rundo and rundolines programs was released. The programs are now precompiled and no longer require installation of AutoIt. On the occasion of the new release, the previous guide to integrating Stata with external text editors was replaced by a new version of the guide.
  • 22 June 2008: Section on troubleshooting added to this guide.
  • 12 October 2008: The rundo and rundolines programs have been confirmed to work with Windows Vista. The acknowledgments section of this guide was expanded.
  • 8 January 2009: The section on known problems and other parts of the guide were revised.
  • 30 March 2009: Version 3.1 of the rundo and rundolines programs was released, with a bugfix. Version 3 of rundo.exe pointed to rundolines.ini instead of rundo.ini. Users who had only installed the rundo program received an error message when they tried to execute a do-file. The code of rundo.exe and rundolines.exe was also updated to version 10.1 of Stata. Both programs continue to work with older versions of Stata.
  • 31 August 2009: INI files updated to support Stata 11.
  • 17 April 2011: INI files updated to support Stata 11.2.
  • 14 August 2011: Version 4 of the rundo and rundolines programs was released. The new programs support Stata 12, released in July 2011. To be compatible with all versions of Stata, it was necessary to add an option to specify the keyboard shortcut to the Stata command window. In Stata versions 8 to 11 the keyboard shortcut is Ctrl+4, in Stata 12 it is Ctrl+1. Users who upgrade from an older version of Stata to Stata 12 must install version 4 of the rundo and rundolines programs. Users of Stata 11 or previous versions of Stata can continue to use version 3.1 of the programs, which have been archived at these links: rundo3.zip (240 KB), rundolines3.zip (240 KB).
  • 21 June 2012: The rundo and rundolines scripts are now hosted at Dropbox. This change was necessary because I no longer maintain a separate website at huebler.info, where I had previously hosted the scripts.
  • 22 December 2013: Version 4.1 of rundo and rundolines was released. The programs are unchanged but the INI files were updated for Stata 13. The guide to integrating Stata and external text editors on this page was also revised and the troubleshooting section was expanded.
  • 18 April 2015: Version 4.2 of rundo and rundolines was released. The programs are unchanged but the INI files were updated for Stata 14. The guide to integrating Stata and external text editors on this page was also revised.
  • 28 June 2015: Version 5.0 of rundo and rundolines was released, with Unicode support for Stata 14. Version 4.2 was withdrawn. Users of Stata 13.1 and older versions of Stata should use version 4.1 of rundo and rundolines. The instructions in this guide were revised.
  • 1 July 2015: Information on Unicode added to sections 4 (Using the programs) and 5 (Known problems and troubleshooting).

Related articles

External links

Friedrich Huebler, 27 April 2008 (edited 1 July 2015), Creative Commons License
Permanent URL: http://huebler.blogspot.com/2008/04/stata.html

13 April 2008

Reported and tested literacy in Nigeria

Household surveys like the Demographic and Health Surveys (DHS) or the Multiple Indicator Cluster Surveys (MICS) collect data on literacy with various methods. One approach is to simply ask respondents whether they can read and write. In an alternative approach, the reading ability is tested by asking respondents to read a sentence from a card. More detailed assessments of literacy, such as the National Assessment of Adult Literacy in the United States, require in-depth questionnaires that are beyond the scope of surveys like the DHS and MICS.

The Nigeria DHS of 2003 collected literacy data with two methods. First, the survey questionnaire contained the question: "Can (name) read and write in any language with understanding?" This question was asked for all household members aged 5 years and older, with two possible answers: yes or no (NPC and ORC Macro 2004: 247). The results of this literacy assessment were described in the article "Adult literacy in Nigeria", published on this site on 5 April 2008.

Second, a randomly selected subsample of women between 15 and 49 years and men between 15 and 59 years were asked to read a card with a simple sentence in their language. If the respondent could not read the whole sentence, the survey staff were asked to probe if the respondent could read any part of the sentence. The result was recorded as one of three options: (1) cannot read at all, (2) able to read only parts of sentence, or (3) able to read whole sentence. If no card in the respondents' language was available, or if the respondent was blind or visually impaired, the result was recorded in a separate category (NPC and ORC Macro 2004: 26-27).

The graph and table below compare the data from the two literacy assessment methods - self-reporting and reading test - for the population aged 15 to 49 years. The self-reported literacy rate, the blue bars in the graph, is the share of respondents that claimed to be able to read and write. The tested literacy rate, the red bars in the graph, is the share of respondents that could read a complete sentence; household members who could only read parts of a sentence were counted as illiterate.

Self-reported and tested literacy in Nigeria, population 15-49 years
Bar graph with self-reported and tested literacy rates in Nigeria
Source: Nigeria Demographic and Health Survey 2003

A comparison of the self-reported and tested literacy rates in Nigeria shows that self-reporting tends to overestimate the degree of literacy in the population. 62 percent of men and women between 15 and 49 years were reported as able to "read and write in any language with understanding". However, the reading test revealed that only 55 percent could read a simple sentence. The difference of 7 percent can be explained by the fact that survey respondents are reluctant to admit that they themselves or other household members, on whose behalf they respond, are unable to read and write. This gap between self-reported and tested literacy can be observed across all groups of respondents: men, women, urban residents, and rural residents. The biggest difference exists in rural areas of Nigeria, were 54 percent of respondents claim to be literate but only 45 percent can in fact read.

Self-reported and tested literacy in Nigeria, population 15-49 years

Self-reported literacy rate (%) Tested literacy rate (%) Difference (%)
Male 74.5 68.4 6.1
Female 50.9 42.8 8.1
Urban 77.8 72.2 5.6
Rural 53.6 45.2 8.4
Total 62.3 54.9 7.4
Source: Nigeria Demographic and Health Survey 2003

References
  • National Population Commission (NPC) [Nigeria] and ORC Macro. 2004. Nigeria Demographic and Health Survey 2003. Calverton, Maryland: National Population Commission and ORC Macro. (Download report, PDF format, 4 MB)
Related articles External linksFriedrich Huebler, 13 April 2008, Creative Commons License
Permanent URL: http://huebler.blogspot.com/2008/04/self-reported-and-tested-literacy-in.html

05 April 2008

Adult literacy in Nigeria

The adult literacy rate is the share of literate persons in the population aged 15 years and older. In Nigeria, a Demographic and Health Survey (DHS) collected data on literacy that can be disaggregated by age, gender, area of residence, and other characteristics. In the survey, conducted in 2003, literacy is defined as the ability to "read and write in any language with understanding" (NPC and ORC Macro 2004: 247). To collect this data, respondents were only asked whether they are literate, but no reading or writing tests were applied. A smaller group of respondents, all women aged 15-49 years and one third of men aged 15-59 years, were asked to read a simple sentence in any of the major language groups of Nigeria (NPC and ORC Macro 2004: 26-27). The literacy rates obtained from this separate reading test were not considered for the analysis that follows, but they are lower than the self-reported literacy rates that cover all survey respondents aged 5 years and older. A separate article on this site describes the difference between self-reported and tested literacy in Nigeria in more detail.

In Nigeria, 55% of the population aged 15 years and older can read and write according to the findings of the DHS. More men (67%) than women (44%) are literate, and the literacy rate is higher in urban (71%) than in rural areas (47%).

Adult literacy rate in Nigeria, 2003
Category
Literacy rate (%)
Male 67.3
Female 43.7
Urban 70.9
Rural 46.8
Total 55.3
Source: Nigeria Demographic and Health Survey 2003

Although almost half of all adult Nigerians cannot read and write, analysis of the survey data by age reveals a steady increase in literacy over the years. Today, more children go to school and learn to read and write than in previous decades. As a result, younger persons are much more likely to be literate than older persons. For the graph below, the survey respondents were divided into 5-year age groups. Among persons aged 15 to 19 years - those who were of primary school age in the 1990s - the literacy rate is 70%. Among persons 80 years or older, only 13% are literate.

A similar rate of increase in the literacy rate can be observed for men, women, urban residents, and rural residents. Among women 80 years or older, only 3% can read and write, compared to 23% of all men in the same age group. Among 15- to 19-year-olds, the female literacy rate is 63% and the male literacy rate 79%. While literacy rates have increased steadily for men and women, there continues to be a large gender gap. A similar gap exists between residents of urban and rural areas.

Adult literacy rate in Nigeria by age, gender and area of residence, 2003
Graph with literacy rates by age, gender and area of residence, Nigeria 2003
Source: Nigeria Demographic and Health Survey 2003

References
  • National Population Commission (NPC) [Nigeria] and ORC Macro. 2004. Nigeria Demographic and Health Survey 2003. Calverton, Maryland: National Population Commission and ORC Macro. (Download report, PDF format, 4 MB)
Related articles

External links

Friedrich Huebler, 5 April 2008 (edited 13 April 2008), Creative Commons License
Permanent URL: http://huebler.blogspot.com/2008/04/adult-literacy-in-nigeria.html