Processing a large dataset of nested data

up vote
0
down vote

favorite

I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:

foo:

- name: "some name"

  location: "some location"

  type: "someType"



  bar:

    - name: "A bar element"

      location: "location here"

      type: "someOtherType"

      attachments:

        - type: "attachmentTypeA"

          name: "Attachment name"

        - type: "attachmentTypeB"

          name: "Attachment name"



  baz:

    - name: "another name"

      location: "another location"

      type: "anotherType"



      qux:

        - name: "My name here"

          location: "My location here"

          type: "SomeOtherTypeHere"



          xyzzy:

            - name: "Another name here"

              location: "Another location here"

              type: "anotherTypeHere"



              bar:

                - name: "Some name here"

                  location: "Some location here"

                  type: "typeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeB"

                      name: "attachment name here"



                - name: "Another name here"

                  location: "Another location here"

                  type: "anotherTypeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeC"

                      name: "attachment name here"

                    - type: "attachmentTypeD"

                      name: "attachment name here"

    - name: "Another baz listing"

      location: "Baz location"

      type: "bazTypeHere"

So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:

foo > baz > qux > xyzzy > bar

However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:

foo

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

As is this:

foo

  baz

    qux

      xyzzy

        bar

          attachments

        bar

          attachments

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

  xyzzy

    bar

      attachments

    bar

      attachments

And so on. It's whacky, I know. But that's the dataset I inherited. I looked at the examples, in particular the DeserializeObjectGraph and LoadingAYamlStream examples. The DeserializeObjectGraph approach gets kind of crazy when the data is laid out like this. I finally gave up on it as it just got too hairy. The stream approach seems like a better fit, I think, but I'm running into troubles.

I am loading up the YAML as follows:

        string contents = System.IO.File.ReadAllText ( fileName );

        var input = new StringReader (contents);

        var yaml = new YamlStream ();

        yaml.Load (input);

As you can see, nothing fancy there. I'm just trying to get a "tree" of objects that I can then iterate through. I tried using the AllNodes property from the root node, but I can't for the life of me figure out how to iterate through them recursively in some manner than makes sense. I will also confess that I am a C# n00btard that is still learning (old C guy here), so bear with me!

Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?

edited Nov 22 at 16:00

asked Nov 22 at 4:24

incursio

What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 at 4:36

This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 at 5:23

Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 at 14:54

Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 at 16:01

I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 at 17:14

|
show 2 more comments

up vote
0
down vote

favorite

I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:

foo:

- name: "some name"

  location: "some location"

  type: "someType"



  bar:

    - name: "A bar element"

      location: "location here"

      type: "someOtherType"

      attachments:

        - type: "attachmentTypeA"

          name: "Attachment name"

        - type: "attachmentTypeB"

          name: "Attachment name"



  baz:

    - name: "another name"

      location: "another location"

      type: "anotherType"



      qux:

        - name: "My name here"

          location: "My location here"

          type: "SomeOtherTypeHere"



          xyzzy:

            - name: "Another name here"

              location: "Another location here"

              type: "anotherTypeHere"



              bar:

                - name: "Some name here"

                  location: "Some location here"

                  type: "typeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeB"

                      name: "attachment name here"



                - name: "Another name here"

                  location: "Another location here"

                  type: "anotherTypeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeC"

                      name: "attachment name here"

                    - type: "attachmentTypeD"

                      name: "attachment name here"

    - name: "Another baz listing"

      location: "Baz location"

      type: "bazTypeHere"

So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:

foo > baz > qux > xyzzy > bar

However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:

foo

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

As is this:

foo

  baz

    qux

      xyzzy

        bar

          attachments

        bar

          attachments

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

  xyzzy

    bar

      attachments

    bar

      attachments

I am loading up the YAML as follows:

        string contents = System.IO.File.ReadAllText ( fileName );

        var input = new StringReader (contents);

        var yaml = new YamlStream ();

        yaml.Load (input);

Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?

edited Nov 22 at 16:00

asked Nov 22 at 4:24

incursio

What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 at 4:36

This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 at 5:23

Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 at 14:54

Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 at 16:01

I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 at 17:14

|
show 2 more comments

up vote
0
down vote

favorite

I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:

foo:

- name: "some name"

  location: "some location"

  type: "someType"



  bar:

    - name: "A bar element"

      location: "location here"

      type: "someOtherType"

      attachments:

        - type: "attachmentTypeA"

          name: "Attachment name"

        - type: "attachmentTypeB"

          name: "Attachment name"



  baz:

    - name: "another name"

      location: "another location"

      type: "anotherType"



      qux:

        - name: "My name here"

          location: "My location here"

          type: "SomeOtherTypeHere"



          xyzzy:

            - name: "Another name here"

              location: "Another location here"

              type: "anotherTypeHere"



              bar:

                - name: "Some name here"

                  location: "Some location here"

                  type: "typeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeB"

                      name: "attachment name here"



                - name: "Another name here"

                  location: "Another location here"

                  type: "anotherTypeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeC"

                      name: "attachment name here"

                    - type: "attachmentTypeD"

                      name: "attachment name here"

    - name: "Another baz listing"

      location: "Baz location"

      type: "bazTypeHere"

So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:

foo > baz > qux > xyzzy > bar

However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:

foo

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

As is this:

foo

  baz

    qux

      xyzzy

        bar

          attachments

        bar

          attachments

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

  xyzzy

    bar

      attachments

    bar

      attachments

I am loading up the YAML as follows:

        string contents = System.IO.File.ReadAllText ( fileName );

        var input = new StringReader (contents);

        var yaml = new YamlStream ();

        yaml.Load (input);

Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?

edited Nov 22 at 16:00

asked Nov 22 at 4:24

incursio

I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:

foo:

- name: "some name"

  location: "some location"

  type: "someType"



  bar:

    - name: "A bar element"

      location: "location here"

      type: "someOtherType"

      attachments:

        - type: "attachmentTypeA"

          name: "Attachment name"

        - type: "attachmentTypeB"

          name: "Attachment name"



  baz:

    - name: "another name"

      location: "another location"

      type: "anotherType"



      qux:

        - name: "My name here"

          location: "My location here"

          type: "SomeOtherTypeHere"



          xyzzy:

            - name: "Another name here"

              location: "Another location here"

              type: "anotherTypeHere"



              bar:

                - name: "Some name here"

                  location: "Some location here"

                  type: "typeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeB"

                      name: "attachment name here"



                - name: "Another name here"

                  location: "Another location here"

                  type: "anotherTypeHere"

                  attachments:

                    - type: "attachmentTypeA"

                      name: "attachment name here"

                    - type: "attachmentTypeC"

                      name: "attachment name here"

                    - type: "attachmentTypeD"

                      name: "attachment name here"

    - name: "Another baz listing"

      location: "Baz location"

      type: "bazTypeHere"

So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:

foo > baz > qux > xyzzy > bar

However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:

foo

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

As is this:

foo

  baz

    qux

      xyzzy

        bar

          attachments

        bar

          attachments

  qux

    xyzzy

      bar

        attachments

      bar

        attachments

  xyzzy

    bar

      attachments

    bar

      attachments

I am loading up the YAML as follows:

        string contents = System.IO.File.ReadAllText ( fileName );

        var input = new StringReader (contents);

        var yaml = new YamlStream ();

        yaml.Load (input);

Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?

yamldotnet

edited Nov 22 at 16:00

asked Nov 22 at 4:24

incursio

edited Nov 22 at 16:00

asked Nov 22 at 4:24

incursio

edited Nov 22 at 16:00

asked Nov 22 at 4:24

incursio

asked Nov 22 at 4:24

incursio

asked Nov 22 at 4:24

incursio

What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 at 4:36

This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 at 5:23

Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 at 14:54

Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 at 16:01

I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 at 17:14

|
show 2 more comments

What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 at 4:36

This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 at 5:23

Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 at 14:54

Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 at 16:01

I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 at 17:14

What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 at 4:36

This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 at 5:23

Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 at 14:54

Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 at 16:01

I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 at 17:14

|
show 2 more comments

active

oldest

votes

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423874%2fprocessing-a-large-dataset-of-nested-data%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

active

oldest

votes

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Some of your past answers have not been well-received, and you're in danger of being blocked from answering.

Please pay close attention to the following guidance:

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Btukfyl