Eating away from home and particularly fast food consumption have been shown to contribute to weight gain. Increased geographic access to fast food outlets and other restaurants may contribute to higher levels of obesity, especially in individuals who rely largely on the local environment for their food purchases. We examined whether fast food and restaurant concentrations are associated with body mass index and whether car ownership might moderate this association. We linked the 2000 US Census data and information on locations of fast food and other restaurants with the Los Angeles Family and Neighborhood Study database, which consists of 2,156 adults sampled from 63 neighborhoods in Los Angeles County. Multilevel modeling was used to estimate associations between body mass index (BMI), fast food and restaurant concentration, and car ownership after adjustment for individual-level factors and socioeconomic characteristics of residential neighborhoods. A high concentration of local restaurants is associated with BMI. Car owners have higher BMIs than non-car owners; however, individuals who do not own cars and reside in areas with a high concentration of fast food outlets have higher BMIs than non-car owners who live in areas with no fast food outlets, approximately 12 lb more (p = 0.02) for an individual with a height of 5 ft. 5 in. Higher restaurant density is associated with higher BMI among local residents. The local fast food environment has a stronger association with BMI for local residents who do not have access to cars.