Research Methods Assignment

From the Empirical Research Model presented by Schwab and discussed in class, what is an important characteristic of a “causal” conceptual relationship included in a theory (we discussed how this characteristic is often missing or neglected).

An important characteristic of a causal conceptual relationship in a theory is external validity. External validity is the ability to generalize the findings from a specific study to other time periods, populations, or cases. Theory assists in addressing generalizability of causal conceptual relationships by making clear why the constructs are causally related, presents evidence for generalizability in a uniform way, assists in determining a wider causal network, and assists in defining constructs, indicating the manner in which measures of constructs may be formulated. Strong causal conceptual relationships from studies should be able to provide actionable insight for future theory.

Why is it important to have a strong relationship between the definition of a concept and how it is operationalized or measured in an empirical study?

The definition of a concept must be clear, precisely defining the variables, in order to create appropriate measures that test what they are intended to test. It reduces subjectivity and increases the reliability of the empirical study. If operational definitions of concepts are not present, researchers may measure insignificant concepts or apply methods in an inconsistent manner.

Assume you have a new measure of construct X. How would you obtain and use evidence of convergent and discriminant validity to support your claim that the new measure should be used.

In order to establish convergent validity, one must indicate that measures that should be related are in actuality related. On the other hand, to establish discriminant validity, one must indicate that measures that should not be related are in actuality not related. One would gather data from a significant sample, then conduct empirical tests to figure out and show that the correlation of the new measure and those that are already established, converge on the same construct X. Then I would introduce another construct into the equation and compare it with construct X, then compare their respective measures to see if their intercorrelations are low. Once convergent and discriminant validity has been proven, I would have again achieved construct validity, with the introduction of this new measure.

If you have an estimated statistical relationship between X and Y (e.g., correlation, information on mean differences across groups), why is it important to eliminate alternative explanations before you claim this estimated relationship is causal and how can you do this?

It is important to eliminate alternative explanations for the relationship between X and Y because one of the internal criteria to prove causation is that there must be no other conceivable opposing conceptual explanation that justify the observed relationship. In order to eliminate alternative explanations, I would conduct a study, composed of a diverse group of individuals which are representative of the population. Then, I would run statistical tests to see if the alternative X correlates with Y and its strength, using the probability of the relationship to statistically validate, and arrive at internal validity. Next, I would figure out if alternative X occurs before Y. The last step would be to think of a reasonable conceptual explanation for their relationship. If I am unsuccessful in proving all these criteria as it pertains to the alternative Xs and Y, then I would conclude that there is a causal relationship between the initial X and Y, as it would fulfil all the criteria as all other explanations have been disproven.

What is a scatter-plot (for a data set with values on X and Y), and how does it change in appearance as the relationship between X and Y gets stronger?

A scatterplot is a visual data representation which indicates how much the dependent variable is affected by the independent variable. The stronger the relationship between the variables, the closer the points (x,y) cluster to the trendline. The value of 1 indicates a perfectly correlated relationship. The trendline goes from lower left to upper right when the relationship is positive, and upper left to lower for a negative relationship.

For a regression model with Y as the DV and multiple Xs as predictors, what are the two key null hypotheses that should be tested, what statistics are used in testing them, and what decision rule is used to test them?

The two key the null hypotheses that are to be tested is that R (multiple correlation coefficient) is equal to 0 [R = 0]. In this, we are using the F-test to determine the overall significance of our regression model for the data. The higher the F-value, the stronger the relationship. If the multiple correlation coefficient (R) is equal to 0, the probability of getting the indicated F-value is indicated by the p-value of significance. If the p-value is less than the model’s significance level α, of 0.05, then we reject the null hypothesis and conclude that the model is significant. But if the p-value is higher than the α, we fail to reject the null hypothesis and conclude that the regression model is not significant.

Once we determine that the model is significant, we then test the null hypothesis β=0, where β is the unstandardized partial regression coefficient to see which predictor or if both predictors are driving the model. We use the T-test to test hypotheses for predictors in isolation. If the t test statistic is greater than the p value, we reject the null hypothesis and conclude that there is a relationship between the DV and the respective IV. Another way in which the null hypothesis can be tested is to look at the confidence intervals of the IVs. If the interval includes 0, then there is a 95% chance that β=0, and we fail to reject the null hypothesis and conclude that there is not a relationship between the DV and particular IV.