Dear all,

The Editorial Board of JAR agreed that by mid July (e.g. by the end of this week) we will have to have a rough idea as to how to proceed in JAR with the reviewing process.
To start the discussion (or better: to continue the discussion; much has already been discussed in Bern, in the Editorial Board meeting and also posted on the JAR Forum) here are some notes which might prompt you to post your thoughts on this issue here in the next 4 days.

- Double blind is not an option
(referees will recognize the ‘signature’ of the author/artist)
- Half blind might be an option; the referees are anonymous, the author/artist not
(advantages: speed of the process; more likely to get external referees; less ‘polite’ reports)
- This could be combined with one aspect of ‘open peer review’: the publication of the anonymous review reports alongside the article/exposition.
- Note that we are dealing in the case of artistic research with ‘extended peer review’, e.g. the peers are both artists and academic (if it still makes sense to make that distinction). It seems to me that a fifty-fifty division between artists and academics is not necessary. The JAR editors have to decide case by case which referees are the best to do the job (a needs-based approach).
- Ad to this half blind, open peer review the review work (pre-reviewing) of a team of dedicated editors (minimum of 2) of JAR, not anonymous, which will be the first step in the reviewing/publication process (after having assessed that the submission fits JAR on minimum material and formal grounds)
- The author/artist has a choice: he or she will adjust the submission/exposition taking the review reports into account, or prefers to publish the submission as it was, pre-reviewed by the JAR editors, alongside the external review reports.
- Here is a crucial role for the editorial board: they will do dedicated work to make the submission into a JAR exposition (which is a good thing in this early stage of the journal).
- After publication JAR might have a section dedicated to non-anonymous post-publication commentary.
- The publication of the review reports and the possibility to have post-publication commentaries will strengthen the idea of artistic research as an ongoing discourse about what counts as (valuable) artistic research.
- One of the advantages of this multilayered way of working (submission – editorial pre-reviewing – anonymous published reviewing – signed commentaries) is that the review process will help, and not frustrate, the authors/artists (and indeed, the artistic research community).
- This could be labelled the ‘ethics of the review process’ (A remark made by Erin Manning of Inflexions at the SLSAeu conference in Riga)

Please post your ideas and/or reaction to these notes in the next 4 days to this Foum.

Yours,
Henk Borgdorff

Re: JAR’s review process
Michael Schwab

I had a meeting with the software developers. Although the reviewing process is not top priority, they wanted to know my/our thoughts on it. This is how it has no been (loosely) proposed:

1. Author has two options: (a) publish: exposition goes straight into the RC, not JAR and is locked for edit; (b) submit to JAR: message can be send to editor in chief; exposition is locked for author; permissions are given to Editor in Chief; exposition is not yet public, but all objects/works are
2. Editor in chief decides on editor and passes permissions to this editor.
3. Editor allows viewing of exposition to reviewers of choice; creates review exposition for reviewers, which reviewers can edit.
4. When a reviewer is done, he presses submit, which locks his review for him and sends a message to editor, who waits for all reviews to have arrived.
5. When this is done, editor gives read permissions for reviews to author and edit permissions on exposition back to author. Reviewer names may be deleted to make review anonymous.
6. Re-submit etc. until all good.
7. Exposition is published.
8. Reviews may be published with or without reviewer names and linked to from TOC. No direct link is planned from exposition since this would require navigation tools that might impact on the design of the exposition.

-> As you might be able to tell, reviews are expositions and should be treated as such not only technically i.e. they can engage in the same visual/media presentation issues as any exposition (‘enhanced reviewing process’?)

Michael

 

Re: JAR’s review process
Stephen Scrivener

The proposal looks good to me and removes certain worrries I had about the workload of editors (although I’m not one). I like the fact that with this approach the editors’ role can be about helping authors to deal with the reviews.

5. …Reviewer names may be deleted to make review anonymous.

My preference, I guess, would be for named reviews, but let’s see what happens and deal with any issues as they arise.

6. Re-submit etc. until all good.

All or the balance is in favour of publication? For example, two reviewers might feel it is fine and one not; (assuming the author decided not to modify the exposition in respose to the negative review) this would be good enough wouldn’t it?

From experience as a journal editor, reviews play an important formative role, i.e, expositions improve with review. With this in mind, I’m not entirely sure what value publishing such reviews would have as they refer to something that is no longer evident. On the other hand, I’m for the reviews being visible as it makes it possible to see what values are operating. I guess the idea of having the initial and final exposition available as locked expositions, one in the RC and the other in JAR, has been considered?

 

Re: JAR’s review process
Michael Schwab
Joined: Sat Jan 02, 2010 2:22 pm
Posts: 73
just to say when I said ‘until all good’ I didn’t want to imply that publication is guaranteed. I should have said ‘until a decision is made’ first by the editor and then by the editor in chief.

The good thing about the proposed design is that review-exposition can but need not be public (or named for that matter), so it’s all down to us to decide how to do it. Personally, I had in mind that both the exposition and the reviews might change during the process and that, further, the review would not simply be ‘about’ an exposition, but also functioning like an exposition as well i.e. engaging with the material in a manner to be expected from all expositions. This is difficult, but perhaps has quite a potential.

Michael

 

Re: JAR’s review process
Stephen Scrivener

>Personally, I had in mind that both the exposition and the reviews might change during the process …

From experience, I’m confident everything will change, at times radically. If there is no intention to publish together with the history of development, then it will be interesting to see what happens. Normally, the last review is “fine, the author’s responded to all issues”. This is obviously not worth publishing. I can see how a different approach might work, but I’m wondering how a decision is reached about publishing or not?

 

Re: JAR’s review process
Michael Schwab

I am a little on the fence on this.

The reason why I thought that a review should be an exposition lies in the fact that a ‘judging distance’ is not assumed. If this is the case, however, we might say that a review does not (only) check, but also engage with the exposition. Then like the exposition, the review would be fully developed at the end of the reviewing process. (As the author responds to questions, the reviewer removes such questioning parts while expanding on the meaning of the author’s choices etc.)

I have no idea if this is feasible, but ultimately the reviews would be ‘engaging’ rather than ‘judging’ – one could say that if no reviewer can be found that wants to engage, an exposition might be questionable. (?) Following this speculation, then everybody on the review board could be asked to respond to an exposition (say max 5 reviewers are chosen) – these reviews would be more engaging. If nobody volunteers, reviewers are asked to do it, as a consequence reviews might be more distanced. (but need not be) [Next to lacks of errors, it is also the excitement and resonances that are being evaluated.]

This is all speculation. I am thinking loudly about possible re-definitions of the reviewer role and an integration into the exposition possibility JAR offers.

Michael

 

Re: JAR’s review process
borgdorff

Dear all,

Here is a proposition for the review process of JAR #0. Please shoot and/or add!
This proposition is partly based on the contributions to the discussion posted on Huddle and on this Forum, especially those of Michael Schwab, George Petelinand Steven Scrivener.
We have to make a distinction between the review process for JAR #0 and the review process for JAR in the future. The proposal is to experiment with different forms of review process for isuue 0, and to communicate this to the ‘world’ with the launch of issue 0 (December).

For JAR #0 I propose three forms:
1. Invited, reviewed by Editorial Board
2. Invited, single blind reviewed
3. Invited, open reviewed
All contributions for JAR #0 are invited. That will be different (possibly) in the future, when we will send out a Call for Submissions.

Ad. 1. This is how it will work with most of the contributions for JAR #0.
Two (or three) editors of JAR assess the contribution, one of them will work closely with the artist/author. The final contribution will also be ‘judged’/discussed by the editorial board (from 15-11-2010 on)

Ad. 2. I propose that some of the invited contributions will go through a single blind review process. For that to happen we have to select, let’s say, three contributions, which – after initial help by dedicated editors – will be reviewed by external referees, to be selected in this stage from our ‘executive working party’. After a positive review the dedicated editor(s) will continue to work with the artist/author on the submission. For this to be successful we need another time frame:
01-11-2010: submission ready (enough) / draft version  to external referees.
15-11-2010: deadline report referees; discussion in Editorial Board; rejection or fine tuning.
NB: Once we have selected the contributions that will undergo this review process, we have to contact the external referees, in order to ask them and to inform them about the time frame. I propose to decide before 20-10-2010 which submissions we choose and which referees we ask.

Ad. 3. For this form (invited, open review) the same time frame as for (2). But here we ask the selected referees to make a report which will be published alongside the submitted exposition, possible in the form of ‘exposition’ in JAR/the research catalogue. They need probably more time to do that. The deadline for their report/exposition is 01-12-2010. We might consider offering a fee here.

Based on our experience with the different forms of review process we can decide later on how to do it in the future.

The dedicated editors, the editorial board and the external referees will use the following criteria:

a. The submitted work claims to be research, not ‘just’ art
b. A description or exposition of the question, issue or problem the research is dealing with.
c. Evidence of innovation in the content, form, or technique of the work in relation to a respected genre of practice.
d. Contextualisation, which includes or might include a discussion of social, artistic and/or theoretical issues that the work responds to, a discussion of a range of positions taken by other artists to which this work contributes a particular perspective, and some documentation of work by the artist that led to the present work.
e. The (kind of) knowledge, understanding, insight, comprehension or experience the research is trying to enhance and convey.
f. The adequacy and soundness of the methods used and thoroughness of research, analysis, and experiment.
g. The potential effectiveness to intellectually engage or to aesthetically affect an intended audience.

As for referees: Note that we are dealing in the case of artistic research with ‘extended peer review’, e.g. the peers are both artists and academic (if it still makes sense to make that distinction). It seems to me that a fifty-fifty division between artists and academics is not necessary. The JAR editors have to decide case by case which referees are the best to do the job (a needs-based approach). For the profile of JAR, however, it is necessary to foreground artists as peers.

Henk

 

Re: JAR’s review process
Michael Schwab

Dear Henk & everybody,

I mostly agree with what you propose – test-run, multiple approaches, practicalities etc. The only thing I am not sure about is the list of criteria. Like testing the kind of reviewing process we want to use, I’d also test the content/criteria/approaches. In the discussion above, I mention for example the review being an exposition in its own right thus not being about an exposition using perhaps different modes of ‘judgment’. Without knowing how this could really work, can we not find a way to find out how it could work outside the criteria you list?

Most importantly, what would people like to see JAR doing?

Michael

 

Re: JAR’s review process
tom_fisher
I think Henk’s 7 criteria are very much to the point – they have in them many of the debates about art as research.

a/ – perhps ‘not just art’ could be ‘not only art’ (‘just’ could read as devaluing).
b/ – yes, a description is necessary
c/ – this one is problematic; whether a body of art involving innovation (or novelty?) is a necessary condition for its being a valid part of a research process is not clear to me. A valid research process could use well-tried art processes (i.e. not innovative ones). Success in the art world does not require being a reflexive researcher (though that is not excluded).
d/ – yes, contextualisation is vital – it is the basis for arguing that the particular art is research
e/ – yes, reflexive understanding of the mode of knowledge production in which the artist is operating is necessary
f/ – adequacy and soundness good.
g/ – potential to communicate; good too.

Thank you, Florian.
If you have read my proposal as ‘checkboxes’ and ‘”good”-markers’ which might ‘guarantee a publication’, I must have expressed myself unclearly… The criteria (I am not so afraid of the word) are not ‘a good-bad-machinery’, but a help for editors and reviewers to assess (if I may use that word) contributions to JAR.
There is a difference, however, between the role of editors and reviewers. I imagine the most supporting work (engaged) will be done by the editors, and we also might expect from our reviewers to engage with the JAR project and the submissions to JAR. But in a different way; the first in working with the artists to come to good submissions, the latter more in the role of assessing the final submissions. The guidelines (maybe that is a better word) are a help in this respect. The ‘assessment’ might take on different forms, e.g. an open review/commentary, but in the end we have to say yes, or no (or maybe, or not yet…)
It would be helpful if you (or others) would focus on the criteria/guidelines in detail. Tell me which one does not satisfy you, and… why? Is nr. 1 to strict, or does nr 4 prohibit any ‘surprise’?
The guidelines are roughly based on what it means to call something ‘research’. In making that claim (this is research) the artist-researchers connects to academia. And although by introducing artistic research into academia we might alter our understanding of what academia is (and I think we do), it also means we connect to what is generaly understood by ‘research’. (We might of course use the word ‘research’ also without any connection to the rest of higher education and research, but what would be the rationale of that? We might as well not use the word at all.)
I agree, ‘artistic research should have the chance – as art should have it – to surprise the reviewer’. But the/a difference between art and artistic research is precisely the idea of ‘research’, isn’t it?

best,
Henk

 

Re: JAR’s review process
FDombois
PostPosted: Tue Dec 14, 2010 7:28 pm
dear henk

guideline sounds good. maybe we can say, that an artistic researcher contributing to JAR will be challenged with questions concerning the aspects, you are giving?
in detail i would see them as follows:

a. The submitted work claims to be research, not ‘just’ art
* yes

b. A description or exposition of the question, issue or problem the research is dealing with.
* i am sure, that often a clear verbal description cannot be given, of what an artistic research (AR) project is dealing with. i would even say, it is interesting, if a work is addressing many questions, if it is open for even more themes, than originally intended etc.
so instead we maybe can say: the researcher should disclose his/her initial interest to do the research, her/his motivation and initial questions. but the reader have than the responsibility not to close down, not to read the work only from the given perspective.

c. Evidence of innovation in the content, form, or technique of the work in relation to a respected genre of practice.
*yes and no. the question: what’s new about it? can be asked. but AR doesn’t need to always be new. i would like to quote here the preface of egon friedell in his “Cultural History of the Modern Age”, where he is focusing on methods like “incompleteness as fas as possible”, “exaggeration” and “the legitimate plagiarist”.

d. Contextualisation, which includes or might include a discussion of social, artistic and/or theoretical issues that the work responds to, a discussion of a range of positions taken by other artists to which this work contributes a particular perspective, and some documentation of work by the artist that led to the present work.
*yes, that’s an interesting point. but again it is the question, how ex- or implicit the artistic researcher needs to demonstrate, that s/he is aware of the context and how much does s/he needs to give explicit connections. i see here also non- or semi-verbal solutions like e.g. the one by thomas hirschhorn (thanks to christoph schenker, from whom i received this picture):
Thank you, Florian.
If you have read my proposal as ‘checkboxes’ and ‘”good”-markers’ which might ‘guarantee a publication’, I must have expressed myself unclearly… The criteria (I am not so afraid of the word) are not ‘a good-bad-machinery’, but a help for editors and reviewers to assess (if I may use that word) contributions to JAR.
There is a difference, however, between the role of editors and reviewers. I imagine the most supporting work (engaged) will be done by the editors, and we also might expect from our reviewers to engage with the JAR project and the submissions to JAR. But in a different way; the first in working with the artists to come to good submissions, the latter more in the role of assessing the final submissions. The guidelines (maybe that is a better word) are a help in this respect. The ‘assessment’ might take on different forms, e.g. an open review/commentary, but in the end we have to say yes, or no (or maybe, or not yet…)
It would be helpful if you (or others) would focus on the criteria/guidelines in detail. Tell me which one does not satisfy you, and… why? Is nr. 1 to strict, or does nr 4 prohibit any ‘surprise’?
The guidelines are roughly based on what it means to call something ‘research’. In making that claim (this is research) the artist-researchers connects to academia. And although by introducing artistic research into academia we might alter our understanding of what academia is (and I think we do), it also means we connect to what is generaly understood by ‘research’. (We might of course use the word ‘research’ also without any connection to the rest of higher education and research, but what would be the rationale of that? We might as well not use the word at all.)
I agree, ‘artistic research should have the chance – as art should have it – to surprise the reviewer’. But the/a difference between art and artistic research is precisely the idea of ‘research’, isn’t it?

best,
Henk

 

Re: JAR’s review process
FDombois
PostPosted: Tue Dec 14, 2010 7:28 pm
dear henk

guideline sounds good. maybe we can say, that an artistic researcher contributing to JAR will be challenged with questions concerning the aspects, you are giving?
in detail i would see them as follows:

a. The submitted work claims to be research, not ‘just’ art
* yes

b. A description or exposition of the question, issue or problem the research is dealing with.
* i am sure, that often a clear verbal description cannot be given, of what an artistic research (AR) project is dealing with. i would even say, it is interesting, if a work is addressing many questions, if it is open for even more themes, than originally intended etc.
so instead we maybe can say: the researcher should disclose his/her initial interest to do the research, her/his motivation and initial questions. but the reader have than the responsibility not to close down, not to read the work only from the given perspective.

c. Evidence of innovation in the content, form, or technique of the work in relation to a respected genre of practice.
*yes and no. the question: what’s new about it? can be asked. but AR doesn’t need to always be new. i would like to quote here the preface of egon friedell in his “Cultural History of the Modern Age”, where he is focusing on methods like “incompleteness as fas as possible”, “exaggeration” and “the legitimate plagiarist”.

d. Contextualisation, which includes or might include a discussion of social, artistic and/or theoretical issues that the work responds to, a discussion of a range of positions taken by other artists to which this work contributes a particular perspective, and some documentation of work by the artist that led to the present work.
*yes, that’s an interesting point. but again it is the question, how ex- or implicit the artistic researcher needs to demonstrate, that s/he is aware of the context and how much does s/he needs to give explicit connections. i see here also non- or semi-verbal solutions like e.g. the one by thomas hirschhorn (thanks to christoph schenker, from whom i received this picture):

File comment: Thomas Hirschhorn: “Where do I stand? What do I want?” (2007)

Hirschhorn_Kanon_Engl-2007.jpg [ 235.81 KiB ]

Dear all,
Here we continue our discussion on JAR’s peer review process, and specifically on the peer review ‘criteria’ (if I may use that word). (See JAR’s Peer Review Guide on www.jar-online.net.)
In the next 10 days we wil discuss together the content and formulations of that Peer Review Guide, which wil help the Editorial Board to come up with a revised document.

To start our online discussion on JAR’s peer review process, let me cite Kathrin Busch from her article: ‘Generating Knowledge in the Arts – A Philosophical Daydream’, published recently in Texte zur Kunst (June 2011) 20/82 (issue: Artistic Research), p. 72:
(Thanks to Michael Schwab for this reference.)

“What has become of the diverse concerms and approaches of a genuinely artistic generation of knowledge under the catchphrase artistic research can be traced as the transformation of an unregulated field into an academic discipline. One consequence of this disciplination is the creation of journals and associations that – brandishing the term artistic research in their titles and names – work to consolidate criteria for assessment of research in the arts. […] In the perspective of the sociology of knowledge, as it were, such rulebooks secure the autonomy of a discipline when it comes to the definition of methods, its objective scope, access restrictions, the awarding of titles, and other criteria of inclusion and exclusion. In the case of artistic research, however, such standardization is conspicuously not the work of artists, who in fact seem to have gradually lost interest in their new discipline. We have little reason, then, to describe the field of artistic research as autonomous, since it is decisively determined by criteria not immanent to art.”

This is an important phrase. Not because it is true, but because it reflects a prejudice or conception of what we do, which we have to take into account.
The whole idea of JAR’s extended peer review process is to assess (which is always tricky) artistic research by several criteria, including those immanent to art.
It seems we have a communication problem. Also therefore it is of utmost importance that we come up with formulations which do justice to our work.

Henk

 

Re: JAR’s review process
borgdorff
PostPosted: Fri Jul 15, 2011 8:19 am
In the same issue of Texte zur Kunst Elke Bippus writes in her article ‘An Aesthiticization of Artistic Research’ (page 100):

“The specificity of artistic research – what distinguishes it from other systems of research – can come to the fore and inform the debate over artistic research and its definitions when aesthetic characteristics, artistic procedures and modes of representation, and their aims are not passed over in favor of an assimilation to scientific metods. [...] The aim, then, must be to bring the peculiar aesthetic qualities of aesthetic research [I quess, this must be: artistic research; HB] into the play of knowledge-production in order to make conflicting procedures, concerns, and positions operative in the field of knowledge production.”

In what follows Elke Bippus refers to Joseph Früchtl’s notion of “epistemological de-foundationalization”; Früchtl: “To have a aesthetic experience, then, means to train the competency to be irritated, the ability to be unsettled, piqued, and challenged.” Bippus: “This competency to be irritated is operative also in artistic research, not only because of its methodological diversity but also because most of the research is presented in the form of exhibitions rather than in writing; that is to say, it is displayed rather than represented.

The question for us is”, of course, how does that translate to the guidelines we want to give to the reviewers of artistic research work?

Henk

 

Re: JAR’s review process
Michael Schwab
PostPosted: Fri Jul 15, 2011 9:57 am
The quote risks a return to a practice/theory divide, which in my opinion we try to leave behind using the concept of ‘exposition’. In short, the choice I have is not between ‘display’ and ‘representation’ but in relation to modes of exposition – both ‘display’ and ‘representation’ are modes of exposition, which might – depending on the context – work and should be supported the framework of JAR. ‘Work’ here means that a group of peers understands the practice as research and considers it to be of high quality.

It is true that some experiences cannot be had in JAR. This is true, however, also for a concert hall and a museum, which have their own deficits. The hope is that JAR like any other framework might allow to present something important of the practice that is exposed – the test will be whether people will find in JAR a suitable way of doing this. If not – nice try…

If we try to assess an exposition, the question need to be: do I think that I can, through an exposition, understand the epistemic dimension of whatever is presented, and, secondly, do I find that this epistemic dimension is interesting, important, exciting?

Michael

 

Re: JAR’s review process
borgdorff
PostPosted: Sat Jul 16, 2011 10:05 am
Dear all,

Attached the draft peer review form.
This a ‘translation’ of the guidelines we are developing into a form reviewers will use to articulate their assessment of JAR submissions.
I propose to take it from here, and comment on this Forum on the review process and ‘criteria’, refering to the formulations in this draft peer review form.

Henk

Attachments:
JAR Peer Review Form (draft) HB 16-7-12.docx [31.1 KiB]
Downloaded 28 times

 

Re: JAR’s review process
Stephen Scrivener
PostPosted: Thu Jul 21, 2011 9:47 am
Dear All

After a struggle with memory and trial and error, I’ve now managed to hit the right username and password combination to join the discussion, but not before I sent a marked up version of the form to Henk. I won’t dwell on minor comments here, leaving Henk to deal with them as he wishes.

My inclination is toward saying less rather in the form rather than more and focussing on the idea of contribution to understanding as the key factor to address as a reviewer. This is more or less as the form starts, although I think that impact is a vague term in this context.

Personally I find the criteria a little heavy handed and not terribly sympathetic to our interests, as in many respects we cannot anticipate what forms of exposition will prove the most convincing and they may turn out to look rather different to conventional academic papers.

In the directions to reviewers, rather than asking reviewers to assess against certain criteria, etc., could we not ask them to explain why they think the exposition does or does not make a contribution to understanding (I’d prefer it if we didn’t make and artistic/intellectual distinction)? If they don’t know how to do this then the criteria might simply function as a checklist. If they do know how to assess an exposition, the criteria might not always fit with the reviewer’s expectations.

On the other hand, I like the pitfalls section, since it acknowledges the fact that there is a debate and things are not settled – it is not dogmatic. I wonder, therefore, whether this section could be expanded to include the criteria, particularly if you are finding that the ‘conventional’ reviewer is proving a hard nut to crack. In other words, the criteria could be presented as valid, but also potential pitfalls in this context.

Finally, I wonder about having a separate evaluation of design and navigation. Surely this contributes to the ‘reading’ of an exposition and will either enhance or diminish meaning. For me, this is bound to form part of the review of the exposition as a contribution to understanding. Again, could this not appear as a pitfall?

 

Re: JAR’s review process
mikaelo
PostPosted: Wed Aug 10, 2011 8:25 pm
Dear All,
I notice that have missed this discussion because of my vacation. I find the peer-reviewing form well structured and very helpful. Especially I like the fact that it is so specific. It helps highlighting the various aspects of the task of the reviewer. Well done!
Best regards,
Mika

[Reposted by admin]