After you. No, really.

27 Jun 2023

No matter where you live or who you are, if you’ve attended school, been on a sports team, played music, or made any kind of presentation ever, there is a core memory that we all share: being chosen to go first. That sinking feeling you get when you stand up and step into the unknown and start speaking, or playing, or demonstrating, or whatever, and all eyes are on you – because you’re the first one. The quiet terror that seeps in as people begin to assess you, the judges peering at you, the knowledge that everyone else has more time to prepare than you…It stays with you forever.

But emotions aside, does it really matter? As long as you are prepared, confident and try your best – your place in the queue is of no consequence, right? Sadly not. It seems when you go is quite significant and – spoiler alert – going first often means you are judged more harshly or receive less credit for your efforts, especially when the selection criteria are stringent and when the evaluators are unfamiliar with the quality of the candidates.

Dr Jiang Bian and Yanbo Wang of the University of Hong Kong, along with two other co-authors, dug deeper into the science of this disagreeable fact using a novel type of research design. Their paper “Good to Go First?” looked at the merits and demerits of “the position effect”, that is, being evaluated at a specific time.

Good to go first? Or not?

During their background research, the authors initially found conflicting evidence as to whether the order in which proposals or performances were evaluated had an effect on the result. In some cases, like with business proposals or investment evaluations, the first presentation was more likely to be approved; also, elections can have “ballot order effect”, whereby people listed first on the ballot win more votes. This is due to the primacy effect, the “tendency for facts, impressions, or items that are presented first to be better learned or remembered than material presented later in the sequence”.

But they also found that in other types of presentations, music performances for instance, or athletic competitions like swimming or ice skating, being evaluated later created an advantage, with one study finding that “figure skaters who perform[ed] later in the first round receive[d] better scores in the first and in the second round” and placed higher overall.

These inconsistencies are due to “thorny inferential challenges”. There is the fact that in situations that allow for self-selection, people who want to go first often do go first, perhaps reflecting their preparation and confidence, which ultimately impacts their results. Also, “the order in which a performance is rendered can have an impact on performance itself”, meaning it’s possible that being randomly selected to go first may induce performance anxiety and thus cause a worse performance – nightmare! The position effect can also affect the perception of evaluators: even if a competition randomly selects performance order, if those evaluating it are more likely to favour or disfavour earlier or later performers…well, we have a problem.

Enter the Innofund

To mitigate these causal identification challenges, the authors chose to study a unique competition: the “Beijing Municipal Innovation Fund for Technology-Based Medium and Small-Size Enterprises”, or Beijing Innofund. The fund’s aim is to promote the success of early-stage technology venture companies by awarding financing to the most promising ones. Each winning company receives a grant of between RMB200,000 to RMB1 million, as determined by the strength of their proposal, with the money helping them “cross the ‘valley of death’” – the precarious time between when a company launches a product and when it begins to generate revenue. An added benefit is that receiving an Innofund grant confers status on the winners, potentially allowing them to secure funding from other sources.

Because the size and prestige of the Beijing Innofund makes it highly competitive and “economically meaningful” for the winners, it attracts thousands of proposals a year. The Innofund’s data set proved to be ideal for other reasons too: grants are judged by individual evaluators working alone, avoiding any peer influence; the evaluations are ranked and funds are allocated based on these rankings, rather than a zero-sum “win or lose” award; marks are recorded after each proposal is reviewed; and the evaluations are based entirely on written materials, with no presentation component – removing any performance anxiety and allowing the authors to see exactly what the evaluators saw.

The authors looked at how the Beijing Innofund’s expert grant evaluators examined almost 3,000 grant proposals as a function of “the position in which they were assigned to evaluate each proposal”. The authors argued that because people assessing serious proposals such as grants “strive to make consistent, reliable and valid evaluations”, in situations where only a small proportion of resource seekers will be selected for funding, grant evaluators generally act conservatively and avoid making “extreme assessments” in their first evaluations by giving out lower marks at the start.

They also proposed that prior experience plays a role: “newbie” evaluators, who have never judged a particular type of competition, tend to rely on a high but abstract “evaluative baseline of quality” at first, while they “calibrate their standards” – especially when the competition is prestigious and selective. This again results in lower scores for those who go first. Evaluators who do have competition-specific experience tend to be more consistently even-handed with their scoring as they have solid understanding of the quality distribution of the resource seekers applying to a specific grant programme.

To test this calibration effect, the authors focused on two types of evaluators: those with professional venture capital (VC) investment experience and those who had judged prior Beijing Innofund competitions. Their hypothesis was that judges with a VC background would have approved higher quality companies in the course of their work, and thus be more critical of the start-ups applying for the Innofund; while the prior Innofund judges, with their specific experience and knowledge about the quality distribution of the grant seekers, would be less likely to be critical of first proposals.

The authors then examined the judging data from the 2016 and 2017 Beijing Innofund competitions. The data set consisted of 400 evaluators and 2,938 grant proposals, with a mean of just over 30 proposals per evaluator. This meant they could accurately measure the “penalty of being first”, since most proposals would not be the evaluator’s first one and since each proposal was evaluated by multiple experts.

Since the Beijing Innofund’s reputation relies on integrity, it uses proprietary software which randomly matches a given proposal with five evaluators in three different roles: one start-up mentor, two technology experts and two finance experts. This added rigour to the study, since a selected proposal is semi-randomly assigned a different priority to each expert – meaning each of the evaluators has no idea what any of the others are looking at. Additional safeguards include performing all evaluations at an off-site hotel, evaluating solely by computer, forbidding all communications during evaluations, and even having anti-corruption officials circulate to ensure adherence to all rules.

After evaluation, the proposals are given a score of between 0 and 100 by each expert, with the final results automatically added up and submitted. The proposal with the highest score receives funding first, the runner-up second, and so on until the fund’s annual budget is finished. For this experiment, the authors included several dummy variables that allowed them to see if a proposal was the reviewer’s first, last or almost first or last. They also created dummy variables to test the evaluators’ professional backgrounds – to determine whether they had VC experience or had been a prior Innofund judge. Since the proposals were evaluated by multiple reviewers, the authors also used a “firm-level fixed effects” model to estimate the position effect in evaluation while holding constant a proposal’s quality.

Bad news for eager beavers

So, after such an intricate set-up and design – what did they find? Unfortunately, the news is still bad for those who like to go first. They concluded without a doubt that “first proposals are evaluated substantially less favourably than those presented thereafter”. With all control variables accounted for and all the statistics checked and rechecked, projects that were evaluated first received overall marks that were over 8% lower than projects evaluated later. Also, as anticipated, the experts with a VC background assigned lower marks across the board, and particularly for their first evaluations – regardless of the quality of their proposal.

And that’s actually quite serious. As the authors note, during “the allocation of funds by VCs, government science grants and corporate [research and development] awards…invariably some proposals must go first or last”. Their experiment concluded that “an applicant that is evaluated first needs [to score] in the top 10^th percentile to merely equal the evaluation of an applicant in the bottom 10^th percentile that is not evaluated first”! This has consequences for both the proposal applicant – who likely misses out on critical funding and a reputational boost; and the grant awarder – who likely misses out on a “potentially favourable investment or candidate”.

This is important information with no simple fix, though the authors have some suggestions. One strategy they found, through research, is to only enter scores after all proposals have been evaluated, but this creates other problems like recall bias. Three intriguing suggestions they offer are 1. Include “placebo proposals” at the start of competitions, which could include random proposals from previous years that will help the judges calibrate their responses; 2. Allow judges to look at several proposals “off the record” to give them an idea of the quality of proposals before they officially begin evaluating; and 3. Make “post-hoc adjustments” in which the software automatically adjusts scores and removes the “first position penalty.”

Regardless, organisations need to take this fact into account to avoid adding injury to insult – as we know, for most of us, it is hard enough to go first, let alone be penalised for it.

About this Research

Jiang Bian, Jason Greenberg, Jizhen Li, Yanbo Wang (2021). Good to Go First? Position Effects in Expert Evaluation of Early-Stage Ventures. Management Science 68(1):300-315.

Read the original article

References

Ballot order effects. (2022, April 20). MIT Election Data + Science Lab. Retrieved April 14, 2023, from https://electionlab.mit.edu/research/ballot-order-effects.

Brooks, A.W., Huang L., Kearney, S.W., Murray, F.E., (2014). “Investors prefer entrepreneurial ventures pitched by attractive men”. Proc. Natl. Acad. Sci. USA 111(12):4427–4431.

Bruine de Bruin, W., (2006). “Save the last dance II: Unwanted serial position effects in figure skating judgments”. Acta Psych. 123:299–311.

Clingingsmith D., Shane S., (2017). “Let others go first: How proposal order affects investor interest in elevator proposals”. Preprint, submitted December 13, 2017. https://osf.io/preprints/socarxiv/6rbyx/.

Greenberg J., (2021). “Social Network Positions, Peer Effects, and Evaluation Updating: An Experimental Test in the Entrepreneurial Context”. Organizational Science. Vol. 32, No. 5. https://doi.org/10.1287/orsc.2020.1416.

Luo, H., (2014). “When to sell your idea: Theory and evidence from the movie industry”. Management Sci. 60(12):3067–3086.

Primacy effect definition. American Psychological Association Dictionary of Psychology. Retrieved April 17, 2023 from https://dictionary.apa.org/primacy-effects.

Unkelbach, C., Memmert, D., (2014). “Serial-position effects in evaluative judgments”. Current Directions Psych. Sci. 23(3): 195–200.

Wang, Y., Li, J., Furman, J.L., (2017). “Firm performance and state innovation funding: Evidence from China’s Innofund program”. Res. Policy 46(6):1142–1161.

Translation

Long held assumptions about the mutually incremental relationship between quantities and discounts have been upended by new research. The rule of thumb that the bigger the purchase quantity, the higher the discount is shown not to hold true for medium-sized customers buying products such as semiconductors, with implications for other products and industries.

Pile them high and sell them cheap. Buy more, save more. These slogans, and the thinking that lies behind them, have been accepted principles of product sales and marketing for generation.

The logic seems indisputable from the points of view of both the seller and manufacturer and that of the buyers. If a seller or manufacturer makes a large number of identical items and a single customer wants to buy a large part of this total production, then that buyer will receive the goods at a cheaper price than a buyer who wishes to buy a much smaller amount of the same product. The accepted theory has been that the seller is eager to dispose of his stock as quickly and as easily as possible, and so a big customer will get a better deal. By the same logic, it follows that customers who buy progressively smaller amounts of the same product will receive progressively smaller discounts.

However, the underlying premise behind these assumptions – that the bigger the purchase, the bigger the discount – has now been shown to be valid for only part of the story. In a new study by Wei ZHANG, Sriram DASU and Reza AHMADI entitled “Higher Prices for Larger Quantities? Nonmonotonic Price-Quantity Relations in B2B Markets,” published in 2017 by the Institute for Operational Research and the Management Sciences in Maryland, USA, the first part of the established belief holds true: the biggest customers do receive the biggest discounts. These customers remain the most valuable to a seller or manufacturer as they account for the bulk of sales. They are therefore typically able to use their size and bandwidth to exert pressure successfully on the seller to get a large discount.

The research focused on investigating the impact of a buyer’s purchase quantity on the discount offered. In this case, the seller was a microprocessor company selling semi-conductors, which are a short-life cycle product. The company negotiates with each of its buyers to set a price for the product. The buyers are mainly large electronic consumer goods manufacturers. In line with established beliefs, the research showed that the discounts received by smaller customers increased in line with the quantities they purchased, and the smaller the quantity they purchased, the smaller the discount they received.

What is unexpected is the experience of medium sized buyers. According to established logic, these customers would be expected to receive bigger discounts on their purchase price than smaller buyers. But this is not the case. In fact, the researchers found that as the quantities bought increase, the discount decreases, and then increases again for the biggest quantities.

“Contrary to our intuition, larger quantities can actually lead to higher prices,” say ZHANG, DASU and AHMADI.Thus, while previous beliefs of a bigger purchase quantity meaning a bigger discount would have resulted in a curve heading steadily north-eastwards, the results of ZHANG, DASU and AHMADI’s studies is an N-shaped curve. This unexpected result is rooted in the importance of capacity to the seller and its impact on the price negotiation process, explain ZHANG, DASU and AHMADI.

To understand the importance of capacity in price setting requires a switch in focus from the buyer’s mind-set to that of the seller. The seller or manufacturer is not concerned solely with getting the best possible price for the product, they also place a value on capacity.

‘’Large buyers accelerate the selling process and small buyers are helpful in consuming the residual capacity,” write ZHANG and his team. “However, satisfying midsized buyers may be costly because supplying these buyers can make it difficult to utilise the remaining capacity, which may be too much for small buyers but not enough for large buyers. Therefore, midsized buyers are charged a “premium.”

To get the best price for all his products, the seller needs to avoid transactions of a medium size and instead plan his sales based on a rationing decision. The rationing decision depends on the remaining capacity level, purchase quantity, demand distribution and the buyer’s profit margin before subtracting the cost of this product. The calculation can be done by following a dynamic capacity rationing formula devised by the researchers. The formula is based on the need for the seller to find a balance between controlling the capacity allocated to each buyer while still offering a capacity range that is acceptable to the buyer.

Ultimately, ZHANG & Co, say, “The seller should reserve capacity for buyers who are willing to pay more.”

The pertinence of the research is clearly of most use to firms manufacturing or selling semi-conductors. This is a highly competitive industry with several unique features and is characterised in particular by fast changing technological developments. In the semi-conductor industry, manufacturing costs are high and lead times are long and these factors lead to inflexible capacities. It is common practice in the industry for sellers to allocate capacity to different product lines based on demand forecasts and to start work on the related production several months ahead of the planned delivery date. Customers arrive sequentially and differ mainly in the quantities of product they order. Although products have a set price, the actual price paid is typically agreed after a process of negotiation, with big buyers usually driving a hard bargain. Because of the nature of the business, negotiation on prices is inevitable, explain the researchers.

“Buyers know that the marginal production cost of microprocessors is low and that sellers are eager to discount prices to fully utilise their capacities. Moreover, buyers can allocate their business among competing sellers.”

But while buyers may have an advantage when it comes to price, sellers often have an advantage when it comes to selling and controlling capacity. Buyers are free to meet their needs by buying from different semiconductor suppliers, but they tend to decide on suppliers early on in the purchasing process. This is because the technical features offered by different suppliers vary, and once selected, these features will impact the design of the buyers’ products and will be difficult and costly to change. That means that buyers tend to keep to their chosen supplier.

The lessons that can be drawn from the study may also be useful to some degree to other businesses and products. Inflexible capacities are also a feature of many businesses in the tourism industry, for example, although the researchers note there are different characteristics and constraints involved – for example, hotel rooms do not go out of date in the same way that semiconductor products become obsolete. Hotel rooms, airline and coach seats are all fixed number items that the seller or owner needs to sell in quantities to his best advantage. The main customers in these industries include bulk buyers such as travel agencies and resellers who want to buy in large quantities but who also want to negotiate the best prices. As in the semi conductor business, the individually agreed deals are closely interconnected, with the price and quantity agreed for one buyer impacting the price and quantity to be agreed for the remaining buyers. The researchers recommend that sellers develop a price-quantity analysis model that can help them optimise their prices. As with semi-conductors, the key point for the seller is the need to control the quantity being sold to each buyer before negotiating the price.

“Basically, given that each transaction has an impact on subsequent transaction, a good model of the price-quantity relation is necessary for the optimisation of the trade-off between the profit from the current buyers and that of future buyers,” they explain.

Contributing Reporter: Liana Cafolla

Source: Wei Zhang, Sriram Dasu, Reza Ahmadi (2017). Higher Prices for Larger Quantities? Nonmonotonic Price–Quantity Relations in B2B Markets. Management Science 63(7): 2108-2126.

https://pubsonline.informs.org/doi/10.1287/mnsc.2016.2454

Share with