Page 1 of 2

problem reading matlab data files

Posted: Mon Apr 22, 2013 11:03 am
by Nabtheberliner
Hi everyone,
I work on book Econometric Modelling with Time Series,specification, estimation and testing by Vance Martin,Stan HUrn, David Harris,Cambridge University Press.
I try to open the datasets with RATS8.0 in order to write the programs and everytime the variables are nameless and empty when i check the series window. It does the same with the xls data files. On the other hand, with another software like Jmulti it works, all the variables exist.
I attach some files here if someone wants to give it a try.
Thanks people.
²

Re: problem reading matlab data files

Posted: Mon Apr 22, 2013 1:59 pm
by TomDoan
Nabtheberliner wrote:Hi everyone,
I work on book Econometric Modelling with Time Series,specification, estimation and testing by Vance Martin,Stan HUrn, David Harris,Cambridge University Press.
I try to open the datasets with RATS8.0 in order to write the programs and everytime the variables are nameless and empty when i check the series window. It does the same with the xls data files. On the other hand, with another software like Jmulti it works, all the variables exist.
I attach some files here if someone wants to give it a try.
Thanks people.
The XLSX file has two sheets, the first of which has the "legend" and the second the data. The data sheet has no labels on it. If you use the data wizard, change to the data sheet, and give it names for the variables, it seems to work fine.

OPEN DATA "sims_data.xlsx"
CALENDAR(M) 1959:1
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS,SHEET="Data") 1959:01 1998:12 y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 $
y12 y13 y14 y15 y16 y17

I imagine you had to do something similar to get it to work with Jmulti.

The Matlab file is a single matrix (ytdata) with 130 rows and 7 columns. I assume the idea is to treat that as 7 separate series. Again, since there is no further information on the file, you'll have to provide the series names. Something like:

OPEN DATA "peersman.mat"
DATA(FORMAT=MATLAB,NOLABELS,SHEET="ytdata") 1 130 y1 y2 y3 y4 y5 y6 y7

Re: problem reading matlab data files

Posted: Tue Apr 23, 2013 7:36 am
by Nabtheberliner
Dear Tom,
The message was sent before i finished it, very sorry. Here is the complete message
Thanks for your answer.

Concerning the sims_data.xlsx file,
indeed it's working,what is strange is that i did the same before getting to you with this matter by renaming the variables as in Jmulti ytdata_1 ytdata_2..... and didn't work??? Also with RATS in the wiard the legend sheet is the one containing the data, and the data sheet contains the legend, again???
OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\sims_data.xlsx"
CALENDAR(Q) 1951:1
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS) 1951:01 2070:04 Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y10 Y11 Y12 Y13 Y14 Y15 $
Y16 Y17

You can notice,i don't have in the line: DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS), what you have SHEET="Data right after ORG=COLUMNS

Also i do the same with another xlsx file from the same book, named G7DATA.xlsx, attached below, and i get the problem even by renaming the variables? The view series shows 0 obs

.

Concerning the matlab file, i do exactly the same
:
OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\peersman.mat"
DATA(FORMAT=MATLAB,NOLABELS) 1951:01 1983:02 Y1 Y2 Y3 Y4 Y5 Y6 Y7

The view series shows nothing,and the output i get is:
## IO9. Invalid input "ytdata" on line 0 while processing series Y1 entry 0
## IO30. There is no series Y2 on the file
## IO30. There is no series Y3 on the file
## IO30. There is no series Y4 on the file
## IO30. There is no series Y5 on the file
## IO30. There is no series Y6 on the file
## IO30. There is no series Y7 on the file


i don't know what to think??
Also with Jmulti there is no need to do anything, it loads everything and the variables are named properly and directly, ready to be used, surprising i have to say

If you have any idea, thanks a lot Tom

Re: problem reading matlab data files

Posted: Tue Apr 23, 2013 9:04 am
by TomDoan
Nabtheberliner wrote:You can notice,i don't have in the line: DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS), what you have SHEET="Data right after ORG=COLUMNS
You need SHEET="Data" because the first sheet on the file is the "Legends" which has no data on it.

Nabtheberliner wrote: Concerning the matlab file, i do exactly the same:
OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\peersman.mat"
DATA(FORMAT=MATLAB,NOLABELS) 1951:01 1983:02 Y1 Y2 Y3 Y4 Y5 Y6 Y7
You need the SHEET="YTDATA" option as shown in my earlier message. The wizard wasn't putting that in---this is a very rare case, most Matlab files with time series data have one matrix per variable rather than a rectangular matrix of unnamed data.

Nabtheberliner wrote: i don't know what to think??
Also with Jmulti there is no need to do anything, it loads everything and the variables are named properly and directly, ready to be used, surprising i have to say
Named what? There are no specific names on either one of those files.
Nabtheberliner wrote: Also i do the same with another xlsx file from the same book, named G7DATA.xlsx, attached below, and i get the problem even by renaming the variables? The view series shows 0 obs
With the G7data file, you have to skip the top two lines (which are descriptions of the series rather than usable series names) and put in the desired names. Using the wizard I get:

OPEN DATA "G7Data.xlsx"
CALENDAR(M) 1960:3
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS,TOP=3) 1960:03 1975:02 cangrowth canspread fragrowth fraspread gergrowth $
gerspread itagrowth itaspread jpngrowth jpnspread ukgrowth ukspread usgrowth usspread

which works fine.

Re: problem reading matlab data files

Posted: Tue Apr 23, 2013 3:02 pm
by Nabtheberliner
Hello Tom,
OK i got it, what i found is with the xlsx files the option SHEET="...", in the cases that you can read below,is not that much needed and it works,
Very strange, the wizard with the sims_data file shows me the legend data and when i open the preview the variables are there. When i swicth to the data, the preview shows the legend and doesn't contain the variables but just the legend, it's reversed but once i know it, it works.
OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\sims_data.xlsx"
CALENDAR(M) 1951:1
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS) 1951:01 1990:12 Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y10 Y11 Y12 Y13 Y14 Y15 $
Y16 Y17


OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\G7Data.xlsx"
CALENDAR(M) 1960:3
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS,TOP=3) 1960:03 1975:02 CAN_Y CANSPREAD FRA_Y FRASPREAD GER_Y GERSPREAD $
ITA_Y ITASPREAD JPN_Y JPNSPREAD UK_Y UKSPREAD US_Y USSPREAD NO


Concerning the matlab files it is needed otherwise indeed it doesn't work, but i load the file first with the wizars then RATS declare there is no variables, next i write the option SHEET="...", and it works
OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\peersman.mat"
CALENDAR(Q) 1960:1
DATA(FORMAT=MATLAB,NOLABELS,SHEET="YTDATA") 1960:01 1992:02 S1 S2 S3 S4 S5 S6 S7

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\portfolio_data.mat"
DATA(FORMAT=MATLAB,NOLABELS,SHEET="YTDATA") 1980:01 2005:02 R1 R2 R3 R4 R5 R6 R7


With JMulti, i load the dataset and the window shows

time/index ytdata_1 ytdata_2 ytdata_3 .................
1960 Q1 ......... ......... ...........
1960 Q2 ......... ......... ...........
1960 Q3 ......... ......... ...........
.
.
.
.
.

THEN I confirm the loading and it works as usual with the variables ytdata_1 ydata_2.......


Thanks a lot Tom

Re: problem reading matlab data files

Posted: Tue Apr 23, 2013 3:34 pm
by TomDoan
I would like to point out that neither the DATA(....) / y1 y2 ... y17 done with RATS nor the ytdata_1, ... that you are getting with JMulti are really giving you data that are ready to go. That's giving you data that are ready for major errors. Give the series real names (as is done in my code for the G7data) and before going on make sure that you have those labels correct. If I had a dollar for each Gauss or Matlab program that I've seen that misidentified data columns, I'd be rich.

Re: problem reading matlab data files

Posted: Tue Apr 23, 2013 4:11 pm
by Nabtheberliner
This is a precious advice!!

[quote][/quote]That's giving you data that are ready for major errors

Could you just precise a bit more what do you mean?
Do you mean, the expressions(letters, number) don't mean anything and by naming them properly we know what we are dealing with?
Thanks Tom

Re: problem reading matlab data files

Posted: Tue Apr 23, 2013 5:03 pm
by TomDoan
Nabtheberliner wrote:This is a precious advice!!
That's giving you data that are ready for major errors

Could you just precise a bit more what do you mean?
Do you mean, the expressions(letters, number) don't mean anything and by naming them properly we know what we are dealing with?
Thanks Tom
Unless Y1 to Y17 are parallel data for 17 countries or 17 firms, there is a very high chance that you will misidentify a data series. I've seen entire papers written with empirical work that assumed that column 5 in their data file was something completely different than what it actually was.

Re: problem reading matlab data files

Posted: Tue Apr 23, 2013 5:33 pm
by Nabtheberliner
Thanks again for the advise Tom, that's clear, i'll be cautious.

Re: problem reading matlab data files

Posted: Wed Apr 24, 2013 12:51 pm
by Nabtheberliner
Hi Tom,
If the data file that you want to open with RATS has no specific SHEET option when you open it with the wiard, like the one i attach here, what do you do in this case? where do you get the information?

Re: problem reading matlab data files

Posted: Wed Apr 24, 2013 12:55 pm
by Nabtheberliner
Sorry, i didn't finish, maybe because they have only one sheet?

Re: problem reading matlab data files

Posted: Wed Apr 24, 2013 1:00 pm
by TomDoan
Nabtheberliner wrote:Sorry, i didn't finish, maybe because they have only one sheet?
Right. There's only the content of the one file. That one is much better than the other files that you've posted because it has well-formatted column labels. The only thing missing are the dates---if you want those, you'll have to assign the start date and calendar scheme yourself.

Re: problem reading matlab data files

Posted: Wed Apr 24, 2013 1:14 pm
by Nabtheberliner
ok i see and it makes completly sens with your remark from yesterday. The thing is that a lot of data files are uncomplete in that sens, badly formatted, one more reason to be cautious.
Thanks Tom

Re: problem reading matlab data files

Posted: Wed Apr 24, 2013 1:34 pm
by TomDoan
Nabtheberliner wrote:ok i see and it makes completly sens with your remark from yesterday. The thing is that a lot of data files are uncomplete in that sens, badly formatted, one more reason to be cautious.
Thanks Tom
Unfortunately, the Martin-Hurn-Harris book is using an eclectic mix of original source data files and in some cases you can't actually tell what's on a file without looking at the Matlab or Gauss source code.

Re: problem reading matlab data files

Posted: Thu Apr 25, 2013 5:05 pm
by Nabtheberliner
Indeed the datasets are mixed up, i try to get the data for the example 13.19 US macro p.485 from what they provide as ressource but when i open it in RATS wizard, i have no clue about what i'm dealing with, so the only way is getting the matlab or gauss source code?
If i wanna delete some columns for instance column 3 and 5, assuming i have all together 8 columns?