Thursday, June 22, 2017

Explanation and critical realism


To explain something is to provide a true account of the causes and circumstances that brought it about. There is of course more to say on the subject, but this is the essential part of the story. And this normative account of explanation should work as well for investigations created within the framework of critical realism as any other scientific framework.

Moreover, CR is well equipped with intellectual resources to produce explanations of social outcomes based on this understanding. In particular, CR emphasizes the reality of causal mechanisms in the social world. To explain a social outcome, then -- perhaps the rise of Trumpism -- we are instructed to identify the causal mechanisms and conditions that were in play such that a novice from reality television would gain the support of millions of voters and win the presidency. So far, so good.

But a good explanation of an outcome is not just a story about mechanisms that might have produced the outcome; instead, we need a true story: these mechanisms existed and occurred, they brought about the outcome, and the outcome would not have occurred in the absence of this combination of mechanisms. Therefore we need to have empirical methods to allow us to evaluate the truth of these hypotheses.

There is also the important and interesting point that Bhaskar makes to the effect that the social world involves open causal configurations, not closed causal configurations. This appears to me to be an important insight into the social world; but it makes the problem of validating causal explanations even more challenging.

This brings us to a point of contact with the theme of much current work in critical realism: a firm opposition to positivism and an allegiance to post-positivism. Because a central thrust of positivism was the demand for substantive empirical confirmation or verification of substantive claims; and that is precisely where we have arrived in this rapid analysis of explanation as well. In fact, it is quite obvious that CR theories and explanations require empirical validation no less than positivistic theories. We cannot dispense with empirical validation and continue to believe we are involved in science.

Put the point another way: there is no possible avenue of validation of substantive explanatory hypotheses that proceeds through purely intuitive or theoretical avenues. At some point a good explanation requires empirical assessment.

For example, it is appealing in the case of Trumpism to attribute Trump's rise to the latent xenophobia of the disaffected lower working class. But is this true? And if true, is it critical as a causal factor in his rise? How would we confirm or disconfirm this hypothetical mechanism? Once again, this brings us into proximity to a few core commitments of empiricism and positivism -- confirmation theory and falsifiability. And yet, a rational adherence to the importance of empirical validation takes us in this direction ineluctably.

It is worth pointing out that the social and historical sciences have indeed developed empirical methods that are both rigorous and distinctive to the domain of the social: process tracing, single-case and small-N studies, comparative analysis, paired comparisons, and the like. So the demand for empirical methods does not imply standard (and simplistic) models of confirmation like the H-D model. What it does imply is that it is imperative to use careful reasoning, detailed observation, and discovery of obscure historical facts to validate one's hypotheses and claims.

Bhaskar addresses these issues in his appendix on the philosophy of science in RTS. He clearly presupposes two things: that rigorous evidence must be used in assessment of explanatory hypotheses in social science; and flat-footed positivism fails in providing an appropriate account of what that empirical reasoning ought to look like. And, as indicated above, the open character of social causation presents the greatest barrier to the positivist approach. Positivism assets that the task of confirmation and refutation concerns only the empirical correspondence between hypothesis and observation.

Elsewhere I have argued for the piecemeal validation of social theories and hypotheses (link). This is possible because we are not forced to adopt the assumption of holism that generally guides philosophy in the consideration of physical theory. Instead, hypotheses about mechanisms and processes can be evaluated and confirmed through numerous independent lines of investigation. Duhem may have been right about physics, but he is not right about our knowledge of the social world.

Sunday, June 18, 2017

Cacophony of the social


Take a typical day in a major city -- a busy street with a subway stop, a park, a coffee bar, and a large consumer financial office. There are several thousand people in view, mostly in ones and twos. Some people are rushing to an appointment with a doctor, a job interview, a drug dealer in the park. A group of young men and women are beginning to chant in a demonstration in the park against a particularly egregious announcement of government policy on contraception.

There is a blooming, buzzing confusion to the scene. And yet there are overlapping forms of order -- pedestrians crossing streets at the crosswalks, surges of suits and ties at certain times of day, snatch and grab artists looking for an unguarded cell phone. The brokers in the financial office are more coordinated in their actions, tasked to generate sales with customers who walk in for service. The demonstrators have assembled from many parts of the city, arriving by subway in the previous hour. Their presence is, of course, coordinated; they were alerted to the demo by a group text from the activist organization they belong to. 

What are the opportunities for social science investigation here? What possibilities exist for explanation of some of the phenomena on display?

For one thing there is an interesting opportunity for ethnographic study presented here. A micro-sociologist or urban anthropologist may find it very interesting to look closely to see what details of dress and behavior are on display. This is the kind of work that sociologists inspired by Erving Goffman have pursued.

Another interesting possibility is to see what coordinated patterns of behavior can be observed. Do people establish eye contact as they pass? Are the suits more visibly comfortable with other suits than with the street people and panhandlers with whom they cross paths? Is there a subtle racial etiquette at work among these urban strangers?

These considerations fall at the "micro" end of the spectrum. But it is clear enough that the snapshots we gain from a few hours on the street also illustrate a number of background features of social structure. There is differentiation among actors in these scenes that reflects various kinds of social inequalities. There are visible inequalities of income and quality of life that can be observed. These inequalities in turn can be associated with current activities -- where the various actors work, how much education they have, what schools they attended, their overall state of health. There are spatial indicators of interest as well -- what kinds of neighborhoods, in what parts of the city, did these various actors wake up in this morning?

And for all of these structural differentiators we can ask the question, what were the social mechanisms and processes that performed the sorting of new-borns into affluent/poor, healthy/sick, well educated/poorly educated, and so forth? In other words, how did social structure impose a stamp on this heterogeneous group of people through their own distinctive histories?

We can also ask a series of questions about social networks and social data about these actors. How large are their personal social networks? What are the characteristics of other individuals within various individual networks? How deep do we need to go before we begin to find overlap across the networks of individuals on the street? This is where big data comes in; Amazon, credit agencies, and Verizon know vastly more about these individuals, their habits, and their networks than a social science researcher is likely to discover through a few hundred interviews. 

I'd like to think this disorderly ensemble of purposive but uncoordinated action by several thousand people is highly representative of the realities of the social world. And this picture in turn gives support to the ontology of heterogeneity and contingency that is a core theme here. 


Wednesday, June 14, 2017

Organizational learning


 

I've posed the question of organizational learning several times in recent months: are there forces that push organizations towards changes leading to improvements in performance over time? Is there a process of organizational evolution in the social world? So where do we stand on this question?

There are only two general theories that would lead us to conclude affirmatively. One is a selection theory. According to this approach, organizations undergo random changes over time, and the environment of action favors those organizations whose changes are functional with respect to performance. The selection theory itself has two variants, depending on how we think about the unit of selection. It might be hypothesized that the firm itself is the unit of selection, so firms survive or fail based on their own fitness. Over time the average level of performance rises through the extinction of low-performance organizations. Or it might be maintained that the unit is at a lower level -- the individual alternative arrangements for performing various kinds of work, which are evaluated and selected on the basis of some metric of performance. On this approach, individual innovations are the object of selection. 

The other large mechanism of organizational learning is quasi-intentional. We postulate that intelligent actors control various aspects of the functioning of an organization; these actors have a set of interests that drive their behavior; and actors fine-tune the arrangements of the organization so as to serve their interests. This is a process I describe as quasi-intentional to convey that the organization itself has no intentionality, but its behavior and arrangements are under the control of a loosely connected set of actors who are individually intentional and purposive. 

In a highly idealized representation of organizations at work, these quasi-intentional processes may indeed push the organization towards higher functioning. Governance processes -- boards of directors, executives -- have a degree of influence over the activities of other actors within and adjacent to the organization, and they are able to push some subordinate behavior in the direction of higher performance and innovation if they have an interest in doing so. And sometimes these governance actors do in fact have an interest in higher performance -- more revenue, less environmental harm, greater safety, gender and racial equity. Under these circumstances it is reasonable to expect that arrangements will be modified to improve performance, and the organization will "evolve".

However, two forms of counter-intentionality arise. The interests of the governing actors are not perfectly aligned with increasing performance. Substantial opportunities for conflict of interest exist at every level, including the executive level (e.g. Enron). So the actions of executives are not always in concert with the goal of improving performance. Second, other actors within the organization are often beyond control of executive actors and are motivated by interests that are quite separate from the goal of increasing performance. Their actions may often lead to status quo performance or even degradation of performance. 

So the question of whether a given organization will change in the direction of higher performance is highly sensitive to (i) the alignment of mission interest and personal interest for executive actors, (ii) the scope of control executive actors are able to exercise over subordinates, and (iii) the strength and pervasiveness of personal interests among subordinates within the organization and the capacity these subordinates have to select and maintain arrangements that favor their interests.

This represents a highly contingent and unpredictable situation for the question of organizational learning. We might regard the question as an ongoing struggle between local private interest and the embodiment of mission-defined interest. And there is no reason at all to believe that this struggle is biased in the direction of enhancement of performance. Some organizations will progress, others will be static, and yet others will decline over time. There is no process of evolution, guided or invisible, that leads inexorably towards improvement of arrangements and performance.

So we might formulate this conclusion in a fairly stark way. If organizations improve in capacity and performance over time in a changing environment, this is entirely the result of intelligent actors undertaking to implement innovations that will lead to these outcomes, at a variety of levels of action within the organization. There is no hidden process that can be expected to generate an evolutionary tendency towards higher organizational performance. 

(The images above are of NASA headquarters and Enron headquarters -- two organizations whose histories reflect the kinds of dysfunctions mentioned here.)


Thursday, June 1, 2017

Social change and leadership


Historians pay a lot of attention to important periods of social change -- the emergence of new political movements, the development of a great city, the end of Jim Crow segregation. There is an inclination to give a lot of weight to the importance of leaders, visionaries, and change-makers in driving these processes to successful outcomes. And, indeed, history correctly records the impact of charismatic and visionary leaders. But consider the larger question: are large social changes amenable to design by a small number of actors?

My inclination is to think that the capacity of calculated design for large, complex social changes is very much more limited than we often imagine. Instead, change more often emerges from the independent strategies and actions of numerous actors, only loosely coordinated with others, and proceeding from their own interests and framing assumptions. The large outcome -- the emergence of Chicago as the major metropolis of the Midwest, the forging of the EU and the monetary union, the coalescence of nationalist movements in France and Germany -- are the resultant of multiple actors and causes. Big outcomes are contingent outcomes of multiple streams of action, mobilization, business decisions, political parties, etc.

There are exceptions, of course. Italy's political history would have been radically different without Mussolini, and the American Civil War would probably have had a different course if Douglas had won the 1860 presidential election. 

But these are exceptions, I believe. More common is the history of Chicago, the surge of right-wing nationalism, or the collapse of the USSR. These are all multi-causal and multi-actor outcomes, and there is no single, unified process of development. And there is no author, no architect, of the outcome. 

So what does this imply about individual leaders and organizations who want to change the social and political environment facing them? Are their aspirations for creating change simply illusions? I don't think so. To deny that single visionaries cannot write the future does not imply they cannot nudge it in a desirable direction. And these effects can indeed alter the future, sometimes in the desired direction. An anti-racist politician can influence voters and institutions in ways that inflect the arc of his or her society in a less racist way. This doesn't permanently solve the problem, but it helps. And with good fortune, other actors will have made similar efforts, and gradually the situation of racism changes. 

This framework for thinking about large social change raises large questions about how we should think about improving the world around us. It seems to imply the importance of local and decentralized social change. We should perhaps adjust our aspirations for social progress around the idea of slow, incremental change through many actors, organizations, and coalitions. As Marx once wrote, "men make their own history, but not in circumstances of their own choosing." And we can add a qualification Marx would not have appreciated: change makers are best advised to construct their plans around long, slow, and incremental change instead of blueprints for unified, utopian change.