Try out a preview of batch build plugins (and other 1.2 features)!

vjau · July 24, 2015, 8:50pm

@glasser on linux, console.log() output in the console comes corrupted. End of lines have character added each time the plugin restart (after a plugin code modification).
The characters seems to come from a platform description (as if there was some sort of buffer overflow), i get things like “owser” (end of browser) or “ux.x86_64” (end of linux.x86_64) if that can help.

jasonxeno · July 24, 2015, 9:05pm

Any idea what this error means that I encountered trying PLUGINS-PREVIEW@1 compared to METEOR@1.1.0.2? I’m not sure how to debug this:

I20150724-15:56:32.498(-5)? Exception in Mongo write: Error: fence has already activated -- too late to add a callback
I20150724-15:56:32.498(-5)?     at [object Object]._.extend.onBeforeFire (packages/ddp-server/writefence.js:69:1)
I20150724-15:56:32.498(-5)?     at Object.callback (packages/mongo/oplog_observe_driver.js:154:1)
I20150724-15:56:32.498(-5)?     at packages/ddp-server/crossbar.js:109:1
I20150724-15:56:32.499(-5)?     at Array.forEach (native)
I20150724-15:56:32.499(-5)?     at Function._.each._.forEach (packages/underscore/underscore.js:105:1)
I20150724-15:56:32.499(-5)?     at [object Object]._.extend.fire (packages/ddp-server/crossbar.js:107:1)
I20150724-15:56:32.499(-5)?     at [object Object].kadira_Crossbar_fire [as fire] (packages/meteorhacks_kadira/packages/meteorhacks_kadira.js:3348:1)
I20150724-15:56:32.499(-5)?     at Object.Meteor.refresh (packages/ddp-server/server_convenience.js:9:1)
I20150724-15:56:32.499(-5)?     at refresh (packages/mongo/mongo_driver.js:349:1)
I20150724-15:56:32.499(-5)?     at packages/mongo/mongo_driver.js:309:1
I20150724-15:56:32.499(-5)?     at runWithEnvironment (packages/meteor/dynamics_nodejs.js:108:1)

glasser · July 24, 2015, 9:13pm

Huh, can you provide a more detailed replication for this? This isn’t something I’ve seen.

glasser · July 24, 2015, 9:18pm

@jasonxeno Similarly, can you provide a more specific reproduction here? This could have something to do with an optimization for Livequery: https://github.com/meteor/meteor/commit/565fa39037b033f1e82ed45a831886e8f355f9fb.

I see kadira mentioned in that stack trace. Can you reproduce this without Kadira? I know Kadira has some hacks in places that we don’t provide hooks, and the commit above could have made a difference. cc @arunoda

arunoda · July 24, 2015, 11:58pm

Actually, this is a marker we add for the CPU profiler. I don’t think that would make a difference.

Here’s the code we used: https://github.com/meteorhacks/kadira/blob/master/lib/hijack/set_labels.js#L110

jasonxeno · July 25, 2015, 2:30am

@glasser Thank you for the hint. I’m getting the same error without Kadira. I’m getting this in a complex area of my code, so I’ll spend some time trying to test simulated isolated pieces of it for a possible reproduction.

In the mean time, I’ll describe what my code is doing in case it helps, the part where the error occurs is when the server makes 5 async HTTP POST calls at the same time, then processes an array of 5 objects from the result of each call, and saves each object as a separate document in a collection in a loop. It errors out after only 1 document is saved after the first HTTP result is returned, and when it tries to save the next document. The whole process goes through several calls and functions.

slava · July 27, 2015, 5:27pm

The new PLUGINS-PREVIEW@2 pre-release has been pushed. The updates from the previous preview:

fixed some importing from imports bugs in Stylus
QA’d and tested on Windows (fixed bugs relevant to handling paths)
all build plugins should prefer using Plugin.fs and Plugin.path that work with the standard paths (/C/some path/file.exe as opposed to C:\some path\files.exe)
fixed import in accounts-ui's less file
a jscript fix for the babel-compiler package
renewed dev-bundle working well for Windows

louis · July 29, 2015, 9:41am

I also ran into an instance where I’m getting the Mongo write fence error. This occurred when testing, using Velocity and Mocha.

Exception in Mongo write: Error: fence has already activated -- too late to add a callback
I20150729-05:24:28.230(-4)?     at [object Object]._.extend.onBeforeFire (packages/ddp-server/writefence.js:69:1)
I20150729-05:24:28.230(-4)?     at Object.callback (packages/mongo/oplog_observe_driver.js:154:1)
I20150729-05:24:28.231(-4)?     at packages/ddp-server/crossbar.js:109:1
I20150729-05:24:28.231(-4)?     at Array.forEach (native)
I20150729-05:24:28.231(-4)?     at Function._.each._.forEach (packages/underscore/underscore.js:105:1)
I20150729-05:24:28.231(-4)?     at [object Object]._.extend.fire (packages/ddp-server/crossbar.js:107:1)
I20150729-05:24:28.231(-4)?     at Object.Meteor.refresh (packages/ddp-server/server_convenience.js:9:1)
I20150729-05:24:28.232(-4)?     at packages/mongo/mongo_driver.js:374:1
I20150729-05:24:28.232(-4)?     at Array.forEach (native)
I20150729-05:24:28.233(-4)?     at Function._.each._.forEach (packages/underscore/underscore.js:105:1)

Here’s a recreation: https://github.com/lourd/meteor-ecmascript-bug

The key file is foo-test.js

describe('Running some tests', () => {
  it('should run', (done) => {
    Meteor.call('dropThings', (err, res) => {
      expect(err).to.equal(undefined);
      expect(res).to.equal(2);
      console.log("test finished");
      done();
    });
  });
});

This will run fine on the client, not triggering any error. But running it on the server throws that Mongo error. Other observations:

It only happens when using Meteor.call in an it, before, or beforeEach block – not if it’s just in a describe block.
It doesn’t happen in the application. Checkout sandbox.js to see that – does the same thing, no problem
Operation on the collection doesn’t matter. I’ve tried removing and inserting, it still throws the same error
The error comes after the test has finished. I’ll see test finished in the console before I see the Mongo write fence error

slava · July 29, 2015, 5:48pm

@louis Let’s open a separate GitHub issue to track this down. It looks relevant to a recently merged optimization to oplog driver.https://github.com/meteor/meteor/issues/4839

trusktr · July 30, 2015, 4:52am

@glasser @slava

Do we need to specify isobuild:compiler-plugin@1.0.0 as a dependency if we use Plugin.registerCompiler? I thought I read that somewhere. Or is that if we’re using the caching-compiler classes?

vjau · July 30, 2015, 5:45am

Yes @joe

trusktr · July 30, 2015, 6:09am

@vjau Ah, good to know. Thanks for the link!

slava · July 30, 2015, 5:49pm

Yes, this is done so older versions of Meteor tool don’t even try to use the new API.

trusktr · August 1, 2015, 6:55pm

@slava @glasser What if the InputFile.add* methods could accept an option called passThrough that would allow other compilers to continue processing files that are marked as passThrough: true from the previous compiler? That might be useful in order to use multiple compilers on the same types of files.

trusktr · August 4, 2015, 11:19pm

Maybe there can be a file inside path/to/your/app/.meteor where we can specify the order of compilers.

timbrandin · August 9, 2015, 5:17pm

This would be great! I’m working on a package that preprocesses js/jsx/html and compiles it into valid React or Angular2 from Blaze.

This package could also be used to extend React and Angular with Handlebars (or Spacebars).

Either to simplify the transitions between the frameworks or for performance comparisons on the same codebase.

But there could also be packages that help pull out translations and replaces the english translation with a valid translation key in a new translation configuration file.

timbrandin · August 9, 2015, 11:32pm

You should check out https://atmospherejs.com/pipeline, I’ve completed some working prototypes with build pipelines for HTML and Ecmascript, allowing build plugin authors access to add to the pipeline and alter the order.

I’ve built it for the latest release of meteor: https://github.com/meteor/meteor/releases/tag/release%2FANUBHAV-ERROR-APP-TEST%401

You can test it out using the latest release, containing a lot of refactors to how templating is done compared to PLUGINS-PREVIEW@1

meteor --release ANUBHAV-ERROR-APP-TEST@1

You will first need to remove templating and ecmascript as they now are in the pipeline instead. Then you can add the pipeline versions.

meteor add pipeline:templating
meteor add pipeline:ecmascript

If you want to extend the pipeline, you can use the exported classes and push in your own compiler like this:

github.com

MeteorPipeline/templating/blob/master/plugin/plugin.js

// The CompileResult type for this CachingCompiler is the return value of
// htmlScanner.scan: a {js, head, body, bodyAttrs} object.
TemplatePipelineCompiler.add('CachingHtmlCompiler', new CachingHtmlCompiler(
  "templating",
  TemplatingTools.scanHtmlForTags,
  TemplatingTools.compileTagsWithSpacebars
));

or this:

github.com

MeteorPipeline/ecmascript/blob/master/plugin/plugin.js

class BabelCompiler {
  processFilesForTarget(inputFiles) {
    inputFiles.forEach(function (inputFile) {
      var source = inputFile.getContentsAsString();
      var inputFilePath = inputFile.getPathInPackage();
      var outputFilePath = inputFile.getPathInPackage();
      var fileOptions = inputFile.getFileOptions();
      var toBeAdded = {
        sourcePath: inputFilePath,
        path: outputFilePath,
        data: source,
        hash: inputFile.getSourceHash(),
        sourceMap: null,
        bare: !! fileOptions.bare
      };

      if (fileOptions.transpile !== false) {
        var babelOptions = Babel.getDefaultOptions();

        babelOptions.sourceMap = true;

This file has been truncated. show original

I hope this inspires someone, for me it feels like a good approach when implementing timbrandin:sideburns@0.3.0 because now you can pass through content to the next one in the pipeline like this (and we don’t have to create a new filename extension).

TemplatePipelineCompiler.push(class Compiler {
  processFilesForTarget(inputFiles) {
    inputFiles.forEach(function (inputFile) {
      let source = inputFile.getContentsAsString();
      let content = source.replace("</template>", "Hello world</template>");
      inputFile.setContentsAsString(content);
    }
  }
});

trusktr · August 17, 2015, 5:05am

Hey guys, how do we make our project use a release permanently so we don’t have to type or alias meteor --release PLUGINS-PREVIEW@2?

glasser · August 17, 2015, 7:11am

meteor update --release PLUGINS-PREVIEW@2

trusktr · August 29, 2015, 1:15pm

Is it possible to tell if an InputFile is a file that has been modified (i.e. is a file that triggered a call to my compiler.processFilesForTarget method)? EDIT: nevermind, I see getSourceHash.

What if there was a boolean property like inputFile._resourceSlot.inputResource.modified that specified if an input file was modified (i.e. the file that triggered a rebuild)? It might be useful when this is implemented: https://github.com/meteor/meteor/issues/4899. That might be more convenient than comparing hashes for some compilers.