eclipse-xtext / eclipse-xtext/xtext

Hidden witespaces rules not followed during model save

Open
#2,525 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help_wanted
Dominant language
Java
Stars
831
Forks
330
Avg merge
3d 7h
Merged PRs (30d)
12

Description

When a witespace hidden rule is removed and introduced the result of Resource.save is invalid for the grammar.

Example:

Model:
	uses += Use*;

SimpleName hidden():
	ID ('.' ID)*
;

MultiName hidden(WS):
	names += BlockName (',' names += BlockName)*
;

MultiQualifiedName hidden():
	(pre = SimpleName '.')? '{' sub = MultiName '}' ('.' post = BlockName)?
;

BlockName:
	name = SimpleName | MultiQualifiedName
;

Use:
	'use' block = BlockName ';'
;

Simple grammar accepting:

use {
	ieee.{std_logic_1164, numeric_std},
	work
}.all;

But not

use {
	ieee.{std_logic_1164, numeric_std},
	work
} . all;

With spaced in between dots.

If I create a test that generates a new BlockName for a post the output is incoherent:

	// Add to each use a post suffix
	def void addPost(BlockName b, String post) {
		if(b instanceof MultiQualifiedName) {
			if(b.post !== null) {
				b.post.addPost(post);
			} else{
				println('Added new')
				val bn = MyDslFactory.eINSTANCE.createBlockName();
				bn.name = post;
				b.post = bn;
			}
		} else {
			println("Modified: " + b.name)
			b.name = (if(b.name !== null) b.name + '.' else '') + post;
		}
	}
	
	@Test
	def void addonBack() {
		val model = '''
			use {
				ieee.{std_logic_1164, numeric_std},
				work
			};
		'''.parse
		validator.assertNoErrors(model)
		
		// Add implicit '.all' suffix
		model.eResource.allContents.toIterable.filter(Use).forEach[
			it.block.addPost('all');
		]
		
		// Transform back to code
		val sBuff = new ByteArrayOutputStream()
		model.eResource.save(sBuff, #{})
		val sReModel = sBuff.toString
		println(sReModel)
		
		// Validate generated code
		val reModel = sReModel.parse
		validator.assertNoErrors(reModel)
		println(EmfFormatter.objToStr(reModel))
	}

This code fail because the newly added node is rendered as:

use {
	ieee.{std_logic_1164, numeric_std},
	work
} . all;

With extra spaces in a context with hidden()

EDIT: fixed small typos in the example

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the supplied grammar and the Resource.save model-mutation test, especially the newly added post node. Inspect how serialization handles hidden whitespace around the inserted dot; done means the saved output reparses without grammar errors and does not introduce the invalid spaces shown in the report.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.