Repetitive Analyses: Cutting Your Workload

Suppose that you need to do an analysis of 200 stock prices, or 25 countries, or some similarly gruesome task. If you’re like us, you want to get it accomplished with as little work as possible. The brute force method—cloning code for each series or country and hand editing it—is a time-consuming, and worse, very inflexible way to handle this.

Fortunately, RATS provides a kit of tools for dealing with these situations. The more structured the analysis, the easier it is to write a short program.

Organizing Your Data

The first step is to save your data in one of the formats where data are read by name—such as RATS format or a spreadsheet. RATS format is better, because you are likely to need to dip into the file many times. We will be changing the name (label) of a series or set of series with each pass, and re-executing DATA, so we can keep refilling the same memory space with new data.

Numbered Series and Arrays of Series

The other key step is to use numbered data series or arrays of series. (See "Series" type for more details). For numbered series, you need to set aside a block of series either with ALLOCATE (using the series parameter), with SCRATCH, or with DECLARE VECT[SERIES]. For instance,

allocate 5 2018:2

creates series 1 to 5. We can reference them as 1, 2, 3, 4 and 5 or with variables such as DO loop indexes which take those values. Note that to use these series we must refer to them explicitly with numbers, integer variables, or names assigned to them with the instruction EQV. This is an older type of coding that you might see in a legacy program.

The more modern use of a VECTOR of SERIES is generally easier and more foolproof than series numbers. For example, instead of the above, do:

allocate 2018:2

dec vector[series] vs(5)

You can refer to specific series using the subscript on VS (and you can use a loop index for that subscript to easily loop over the set of series). You can also use the name of the vector by itself in place of a list of series when you want an operation (such as PRINT or COPY) to apply to all series in the VECTOR. For example:

print / vs(1) Prints the series stored in element 1 of VS

print / vs Prints all 5 series stored in VS

Using LABEL Variables and the %S Function

LABEL variables and VECTORS of LABELS play an important role in these examples. The LABELS instruction resets the label of a series, which, as mentioned above, is what the DATA instruction uses to locate data on a RATS format or spreadsheet file.

“Labels” are strings of up to sixteen characters. You can concatenate them using either the explicit function %CONCAT(a,b), or simply by using a+b. The a+b notation also works if b is an integer or integer-valued expression. The function %S(label) can be used to refer to (or create) a series with the given label. Examples of the use of labels:

declare label lab lab1 lab2

compute lab="abcde"

compute lab1=lab+lab lab1="abcdeabcde"

compute lab3=lab+3 lab3="abcde3"

set %s(lab3) = %s(lab)^3 Series ABCDE3 = ABCDE^3

Examples

VECTORS of LABELS are particularly useful for reading in or storing lists of variable or file names. For example, suppose you have a text file containing a list of (an unknown number of) series names, and you want to read those series in from a data file. You could do something like:

declare vector[label] names

open data namelist.txt

read(varying) names

compute num = %size(names)

declare vector[series] vars(num)

labels vars

# names

open data data.xls

data(format=xls,org=cols) / vars

This pulls in all the variable names from namelist.txt, determines how many names were read in, creates a VECTOR of SERIES of that size, and assigns the labels to the series in the vectors. It then reads in those series from a spreadsheet (RATS looks for series on the file whose names match the labels of the VARS series).

Example BETAS.RPF computes beta values for a group of stocks. First, consider computing a beta value for a single security, which requires regressing the return for the security on the market return. Let’s let MARKET be the market return and SECURITY be the return on the single security. We just regress series SECURITY on MARKET and report the result:

linreg(noprint) security

# constant market

?##.#### %beta(2)

The tickers for the stocks are on a separate file with the symbols separated by blanks. The actual data is on a RATS format file. The program runs over a loop on the tickers, processing a single security per pass, sending the information to a separate file using DISPLAY with the UNIT=COPY option.

cal(m) 1986:1

allocate 1996:12

open data ticker.txt

declare vector[label] tickers

read(varying) tickers

compute ntickers=%size(tickers)

open data returns.rat

open copy betas.lst

data(format=rats) / market

do i=1,ntickers

clear security

data(format=rats) / security<<&tickers(i)

linreg(noprint) security

# constant market

display(unit=copy) tickers(i) @20 ##.##### %beta(2)

end do i

close copy

INFLUNEM.RPF does an overlay graph of unemployment and inflation for seven countries. The raw series on the data file have labels which are generated by appending SUR (Standardized Unemployment Rate) and DEFLS suffixes to three letter country prefixes. The INQUIRE instruction determines the maximum range over which both inflation and unemployment are defined. This is example .

open data oecddata.rat

cal(q) 1960:1

all 1998:4

declare label country

dofor country = "can" "deu" "fra" "gbr" "ita" "jpn" "usa"

clear price sur

data(format=rats) / sur<<&country+"sur" price<<&country+"defls"

set inflation = 400.0*log(price/price{1})

inquire(regressorlist) n1 n2

# sur inflation

graph(header="Inflation vs Unemployment for "+%strupper(country),overlay=line) 2

# sur n1 n2

# inflation n1 n2

end dofor

HASH Aggregator

The HASH aggregator can be very handy in these types of programs. For instance, the INFLUNEM.RPF example has graphs labeled with the abbreviation of the country. If you want long names instead, you can add something like:

dec hash[string] longnames

compute longnames("can")="Canada"

compute longnames("deu")="Germany"

etc.

then on the GRAPH inside the loop, use longnames(country) rather than %strupper(country) to use the long form of the name.