Memory Problem

[chibuzor95]

I am having problem running a logistic regression in rats 7.3. The error states M4. A memory request for an additional 672928452 bytes cannot be satisfied. I will appreciate any help in resolving this problem.

Regards,

Okechukwu

[TomDoan]

You will have to include your program (paste it in, select and hit the code button). There's no way we can help with a memory overrun without seeing the size of the problem.

[chibuzor95]

Code: Select all

all 102353
open data drc2003nsch1 data.xls
data(format=xls,org=cols) / IDNUMR STATE MSA_STAT AGEYR_CHILD TOTKIDS4 AGEPOS4 S1Q01 RELATION TOTADULT3 EDUCATIONR $
 PLANGUAGE S2Q01 S2Q02R HGHT_FLG S2Q03R WGHT_FLG BMICLASS S2Q04 S2Q05 S2Q06 S2Q07 S2Q08 S2Q09 S2Q10 S2Q11 S2Q12 S2Q13 $
  S7Q34 S7Q35 S7Q36 S7Q37 S7Q38 S7Q39 S7Q40 S7Q56 S7Q45 S7Q53 S7Q52 S7Q44 S7Q41 S7Q54 S7Q59 S7Q48 S7Q62 S7Q63 S8Q01R S8Q03 $
  S8Q02R S8Q04 S8Q05 S8Q06 S8Q07 S8Q08 S8Q09 S8Q10 S8Q11 S8Q12 S8Q13 S8Q14 S8Q15 FAMSTRUCT S9Q05R S9Q08 S9Q09 S9Q10 S9Q18 $
  S9Q19 S9Q20 S9Q15 S9Q15A S9Q15B S9Q15C S9Q15D S9Q15E S9Q11B S10Q01 S10Q02 S10Q03 S10Q04 S10Q05 S10Q06 S10Q07 S10Q08 $
  S11Q01 RACER RACEAIAN RACEASIA RACE_HI S11Q03 S11Q04 S11Q05 S11Q06R S11Q08 POVERTY_LEVELR C11Q11 C11Q11A C11Q11B S9Q34 $
  WEIGHT_I age_3 age_5 sex race_5 povlev_4 rxmeds serve func therapy mhealth lwiscren def_4 metlwi hp2010_1 hp2010_2 $
  hp2010_3 hp2010_4 hp2010_5 hp2010_6 ind1_1 ind1_2 ind1_3 ind1_4 ind1_5 ind1_6 ind1_7 ind1_8 ind1_9 ind1_10 ind1_10a $
  ind1_11 ind1_12 ind2_1 ind2_2 ind2_3 ind2_4 ind2_5 ind2_6 ind2_7 ind2_8 ind3_1 ind3_2 ind3_3 ind4_1 ind4_2 ind4_3 ind4_4 $
  ind4_5 ind4_6 ind4_7 ind4_8 ind4_9 ind4_10 ind4_11 ind4_12 ind4_13 ind4_14 ind4_15 ind5_1 ind5_2 ind5_3 ind5_4 ind5_5 $
  ind5_6 ind5_7 ind6_1 ind6_2 ind6_3 ind6_4 ind6_5 ind6_6 ind6_7 ind6_8 ind6_9 ind6_10 ind6_11 ind6_12 ind6_13 ind6_14 $
  ind6_15 ind6_16 ind7_1 ind7_2 ind7_3 S7Q26r2 S7Q26r3
*
set mississippi = state==26
set NHS = EDUCATIONR<1
set HS = EDUCATIONR==2
set MHS = EDUCATIONR>3
set Ehealth = S2Q01==1
set Vhealth = S2Q01==2
set Ghealth = S2Q01==3
set Fhealth = S2Q01==4
set Phealth = S2Q01==5
set male = S1Q01==1
set female = S1Q01==2
set underweight = BMICLASS==1
set normalweight = BMICLASS==2
set Roverweight = BMICLASS==3
set overweight = BMICLASS==4
set tpbiologica = FAMSTRUCT==1
set tpstepfam = FAMSTRUCT==2
set smothernf = FAMSTRUCT==3
set other = FAMSTRUCT==4
set emhealth = S9Q08==1
set vgmhealth = S9Q08==2
set gmhealth = S9Q08==3
set fmhealth = S9Q08==4
set pmhealth = S9Q08==5
set efhealth = S9Q09==1
set vgfhealth = S9Q09==2
set gfhealth = S9Q09==3
set ffhealth = S9Q09==4
set pfhealth = S9Q09==5
set nhhsmoke = ind6_4==0
set shhsmoke = ind6_4==1
set nfemployed = S11Q08==0
set femployed = S11Q08==1
set fpl1 = povlev_4==1
set fpl2 = povlev_4==2
set fpl3 = povlev_4==3
set fpl4 = povlev_4==4
set dnlwkpoorhh = ind6_5==1
set lwkpoorhh = ind6_5==2
set evmmentalh = ind6_2==1
set gmmentalh = ind6_2==2
set fpmmentalh = ind6_2==3
set dplayosports = ind5_3==0
set playosports = ind5_3==1
set dnattdheadst = ind5_1==0
set attdheadst = ind5_1==1
set childwconshins = ind3_2==0
set childwoconshins = ind3_2==1
set childnoins = ind3_1==0
set childins = ind3_1==1
set nchildbreastfed = ind1_3==0
set childbreastfed = ind1_3==1
set evchildteeth = ind1_2==1
set gchildteeth = ind1_2==2
set fpchildteeth = ind1_2==3
set childusborn = S11Q05==1
set childnusborn = S11Q05==0
set hisorgin = S11Q01==1
set nhisorgin = S11Q01==0
set white = RACER==1
set black = RACER==2
set multirace = RACER==3
set other = RACER==4
set healthcov = S9Q15E==1
set nhealthcov = S9Q15E==0
*
linreg childwconshins
# constant male female white black hisorgin mississippi nhs hs mhs ehealth Vhealth ghealth fhealth phealth $
  underweight normalweight Roverweight overweight tpbiologica tpstepfam smothernf other nfemployed femployed $
 dplayosports playosports dnattdheadst attdheadst nhhsmoke shhsmoke fpl1 fpl2 fpl3 fpl4 dnlwkpoorhh lwkpoorhh
*
ddv(dist=logit) childwconshins
# constant male female white black hisorgin mississippi nhs hs mhs ehealth Vhealth ghealth fhealth phealth $
  underweight normalweight Roverweight overweight tpbiologica tpstepfam smothernf other nfemployed femployed $
 dplayosports playosports dnattdheadst attdheadst nhhsmoke shhsmoke fpl1 fpl2 fpl3 fpl4 dnlwkpoorhh lwkpoorhh
*
ddv(dist=probit) childwconshins
# constant male female white black hisorgin mississippi nhs hs mhs ehealth Vhealth ghealth fhealth phealth $
  underweight normalweight Roverweight overweight tpbiologica tpstepfam smothernf other nfemployed femployed $
 dplayosports playosports dnattdheadst attdheadst nhhsmoke shhsmoke fpl1 fpl2 fpl3 fpl4 dnlwkpoorhh lwkpoorhh
*
* Test whether "race" dummies are significant.
* Use "Wald" test first.
*
exclude(Title='wald test of race dummies')
# white black hisorgin
compute logunres = %logl
*
ddv(dist=logit) childwconshins
# constant male female mississippi nhs hs mhs ehealth Vhealth ghealth fhealth phealth $
  underweight normalweight Roverweight overweight tpbiologica tpstepfam smothernf other nfemployed femployed $
 dplayosports playosports dnattdheadst attdheadst nhhsmoke shhsmoke fpl1 fpl2 fpl3 fpl4 dnlwkpoorhh lwkpoorhh
compute logres=%logl
*
compute lratio=2*(logunres-logres)
cdf(Title='LR Test for race Dummies') chisqr lratio 3
*
table

[TomDoan]

That's a really big file. In fact, it's so big, I don't know how you manage to create it. The specs for Excel 2003 are:

65,536 rows by 256 columns

(that's from the Microsoft web site). I'm fairly sure that your problem isn't with the regressions, but with the data step. What was the format of your original data source?

[chibuzor95]

The original data was in spss and i transformed it to comma delimnated and excel 2007 formats.

regards,

Okechukwu

[TomDoan]

You might not be able to read that as Excel 2007; Excel 2007 has so many layers between the file and the actual data that it's possible that you might run out of memory processing such a large file. Excel 2003 won't work because it can't handle that many records. However, you shouldn't have any problems with the comma separated file.

[chibuzor95]

Thanks Tom.